[FLASH-USERS] MPI bug on SGI

Hi!

I belive I have just encountered a bug in at least one FLASH routine.

Our System:

SGI origin IRIX64 6.5 07091542 IP35
MPI 3.2.0.7 (MPT 1.4) -mpi Version 1266208220
MIPSpro Compilers: Version 7.4

Problem:

3-D hydro starting with 5x5x3 blocks, maxref = 3
we get a:

*** MPI has run out of request entries.
*** The current allocation level is:
*** MPI_REQUEST_MAX = 16384
IOT Trap
MPI: MPI_COMM_WORLD rank 0 has terminated without calling MPI_Finalize()
MPI: aborting job

when:

 [04-17-2003 18:48.39] <<< refined: tot_blocks = 2723 >>>
 [04-17-2003 18:48.59] *** plot file: gas_pre_hdf_plt_cnt_0009
 [04-17-2003 18:49.08] *** wrote to gas_pre_hdf_plt_cnt_0009
 [04-17-2003 18:49.41] step 91 t= 2.664610E-06 dt= 1.431909E-08
 [04-17-2003 18:50.12] step 92 t= 2.693248E-06 dt= 1.426362E-08

MPI_REQUEST_MAX = 16384 is a hard limit on our machine and can only be
decreased using the enivronment variable.

I tracked that down to at least one non-blocking send to a blocking receive
where the send reqest was not freed. This happens at least is:

line 241 AMR_redist_blk.F90 FLASH 2.2 & 2.1

according to the MPI standard that should be fixed by having a MPI_WAIT
following the non-blocking send immediately.

I attached a test program to this email which illustrates this point (at
least on our system). Version as supplied should fails, if you uncomment the
MPI_wait it should run until forever.

Please let me know what you think about this and if I am right, if there are
more if these problems in the code (I presume you would know from the top of
your head). I try to get through it tonight, but Flash is a little bit
bigger, so help would be appreciated.

Regards,

Markus.

-- 
_______________________________________________________________
Markus Gross AMIMechE BEng (Hons.) Mechanical Engineering
Heriot Watt University Edinburgh
Department of Mechanical and Chemical Engineering
Associate member of the Institution of Mechanical Engineers
Member of the SPIE
_______________________________________________________________
further contact:
Phone   : +44 (0) 131 449 5111 etx. 4737
UNiX talk: talk markus@lasersim.mce.hw.ac.uk
_______________________________________________________________
"Plans are a place to begin," Grove said. "They rarely deliver
you to where you expect. Make your plans knowing you are going
to throw them away."
_______________________________________________________________
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
mQGiBDt30nARBACUL2XadcLU9YS+3ndnWUUpnrhAD3Ab/TqJP1Obtz7H356+vigu
2m3exw2P9fky08SBdTbjOCV5OTmVcRW5F0SNMTIl8gXtGdIcQHCcZxi1GcRJtTBj
npfRCmZojHr8YgsKDdJ9xupNm+UYWcg1BpFX7xe65kXM4oVwVyuCcUT+VwCg6/Hd
6oVoRHm1eq1NdMbCfsc9x78D/2uGQSMJBKewQ/uSPpdEg+bkAQo5Y7MooEeYpLk9
Iroo4rqxp12k0+bWHWVMdfywI80tivorTIgBTKpCtjLaiKen9E9zECo+2OR4+XRQ
O6cBKXOm+V2lJOpFYOh500tJHPggshD21jksNPKJr9CM1byeZFw38fFJez+DyShZ
Gmc6A/9L21jqbVjxvAPwaqe09GEmpT1EYOWtcPYSh2cuo1v6XsGFXfi8tnRSizhH
xsyIfuOUuj0NKRIcXH60awrvABSa45fvQUefPw0hxSwdCnfc0yEqcD4x/03xTZJS
1hUK3T4oTlrXNugaCb59ezciEIh2y/d8pW1DR3RMcgXUSz39tLQhTWFya3VzIEdy
b3NzIDxtLnMuZ3Jvc3NAaHcuYWMudWs+iFcEExECABcFAjt30nAFCwcKAwQDFQMC
AxYCAQIXgAAKCRBTnmsr4a/0+e3tAJoDk08MfeqB83FXP6g8QlDyTlotSwCgiKwS
PPkVAbopX8mahpkrphEaunW5Ag0EO3fSghAIAOeLTMjdFkl/xBvjgD2N0n4DCK4N
EVfaXmkJK5dbTM3K6ZsJjILZ/TfC6R8oxYPsYKwkwLJJEe4E/h1TFg7ogDYbViW8
DihOyeN8zKSymLTbbikEJX8aNeCdBRuy87X0pBazDq6//+oTIatS2EsvMyKk0tEZ
Mh0vTmUJqoeorQeAKH66lPsH2A8QI6u0zv17t4S2aLNBXIbKgOJwsglZ1IwjwowR
KEGsLsHInNkP4bnlBjd51RMiJT979QMPmNgLGEId9Vi0oijtF/v6J91GtH++G6WE
HM08DsP/BKjRgO9ivWP7lOHSDCJpGYti5K6g38JAc1H2EoHGLD7eRgI/kR8AAwUH
/j6NVGLxnFGDHGcBXEb2Y4Wwhf/eUeE8Wit0qqRoNypBuzvWl96GXSPj/saQA/Ri
XcFWR1G3E4V/zdxYnU/xvvGADj9jKHpZjAs73fJJxLc85gzHLcx00tHcQVA07Ihr
hao8WY+BtKhkZorYEJEvSDJ9xufjL/o1t40eCsHQ3b9Jh/1lqQ2KT1lhDXAaDo1P
pE38E2uImC0umIW/UtuKJ1k52nXx2qulfcTjAHd35XidwGreH3aHy3fV+x/+wW1T
CQBHy27dG7JL3ESzPiKW0w2miWjtd4k5jqJTfgKI4wFUhAShxTjp8M051ArzopRJ
iX+s7RrHuEXHjh45V1zU5BWIRgQYEQIABgUCO3fSggAKCRBTnmsr4a/0+U/rAJ9o
lOhPGltVnRncYJPzyIk00H0JDQCfWrlnYDh5cRQmhnHilnrVO5Ykr24=
=i3eC
-----END PGP PUBLIC KEY BLOCK-----