[FLASH-USERS] problem with FLASH2.5/source/mesh/amr/init_mesh.F90 on zeus.llnl.gov
Christopher W. Mauche
mauche at cygnus.llnl.gov
Tue May 8 13:25:51 CDT 2007
Dear fellow FLASHers,
I just downloaded a new clean version of flash v2.5 to zeus.llnl.gov, and
attempted to test/verify it by running the sedov test problem. I was able to
unpack and compile the code, but find an error when I run it. Specifically:
> ./setup sedov -auto
> gmake
> cd object
> cp ../setups/sedov/flash.par flash.par
> srun -N1 -n2 -p pdebug flash2
"get_parm_from_context: igeomx,igeomy,igeomz: 4599616371426034975
4621819117588971520 4591870180066957722"
ERROR: invalid mesh geometry, check igeom{x,y,z}
"get_parm_from_context: igeomx,igeomy,igeomz: -1 -1 -1"
[1] [MPI Abort by user] Aborting Program!
srun: mvapich: May 08 11:18:01: ABORT from MPI rank 1 [on zeus13]
srun: error: zeus13: task1: Exited with exit code 1
srun: error: zeus13: task0: Killed
where the lines in quotes are from a print statement I added to
FLASH2.5/source/mesh/amr/init_mesh.F90 [just below the line
"call get_parm_from_context(GLOBAL_PARM_CONTEXT, "igeomz", igeomz)"],
which is the source of the error. It appears that igeom{x,y,z} is read in
properly by one of the processors, but not by the second one.
If I restrict the run to one processor, I get a different error:
> srun -N1 -n1 -p pdebug flash2
"get_parm_from_context: igeomx,igeomy,igeomz: -1 -1 -1"
flash: initializing for sedov problem.
p_ambient = 1.00000000000000008E-5
rho_ambient= 1.
gamma = 1.3999999999999999
exp_energy = 1.
r_init = 1.3671875E-2
p_exp = 681.16756574500903
xctr = 0.5
yctr = 0.5
zctr = 0.5
ndim = 2
nsubzones = 7
0 - MPI_SSEND : Invalid rank 1
[0] [] Aborting Program!
srun: mvapich: May 08 11:23:39: ABORT from MPI rank 0 [on zeus13]
srun: error: zeus13: task0: Exited with exit code 70
Does anyone have an idea what might be causing these problems, and what I can do
about them? Please note that Flash v2.5 _used_ to work for me on zeus, but now
it fails with these errors.
Thanks,
Chris Mauche
PS: For the record, my FLASH2.5/source/sites/zeus.llnl.gov/Makefile.h is:
# FLASH makefile definitions for the 64 bit Intel icc/ifort compiler on Linux
# Created for zeus.llnl.gov by nathan and cal.
#
# Fedora Core 3
# Intel Linux Fortran 9.0
# hdf5-1.6.5 (icc90)
# mpich-1.2.7 (icc90)
#----------------------------------------------------------------------------
# Set the HDF/HDF5 library paths -- these need to be updated for your system
#----------------------------------------------------------------------------
MPI_PATH =
HDF4_PATH =
HDF5_PATH =
/usr/global/tools/hdf5/installs/chaos_3_x86_64_ib/hdf5-1.6.5/parallel
ZLIB_PATH =
PAPI_PATH =
PAPI_FLAGS =
NCMPI_PATH =
MPE_PATH =
#----------------------------------------------------------------------------
# Compiler and linker commands
#
# We use the f90 compiler as the linker, so some C libraries may explicitly
# need to be added into the link line.
#----------------------------------------------------------------------------
FCOMP = mpipathf90
CCOMP = mpipathcc
CPPCOMP = mpipathCC
LINK = mpipathf90
# pre-processor flag
PP = -D
#-----------------------------------------------------------------------------
# Compilation flags
#
# Three sets of compilation/linking flags are defined: one for optimized code
# code ("-opt"), one for debugging ("-debug"), and one for testing ("-test").
# Passing these flags to the setup script will cause the value associated with
# the corresponding keys (i.e. those ending in "_OPT", "_DEBUG", or "_TEST") to
# be incorporated into the final Makefile. For example, passing "-opt" to the
# setup script will cause the flags following "FFLAGS_OPT" to be assigned to
# "FFLAGS" in the final Makefile. If none of these flags passed, the default
# behavior will match that of the "-opt" flag.
# In general, "-opt" is meant to optimize compilation and linking. "-debug"
# should enable runtime bounds checking, debugger symbols, and other compiler-
# specific debugging options. "-test" is useful for testing different
# combinations of compiler flags particular to your individual system.
#----------------------------------------------------------------------------
FFLAGS_OPT = -c -r8 -i8 -m64 -march=opteron -O3 -g
FFLAGS_DEBUG = -c -g -r8 -i4 -check bounds -check format -check
output_conversion -warn all -real_size 64
FFLAGS_TEST = -c -r8 -i4 -O2 -real_size 64
CFLAGS_OPT = -c -O3 -r8 -i8 -m64 -march=opteron -g
CFLAGS_DEBUG = -c -g -debug extended -D_LARGEFILE64_SOURCE
CFLAGS_TEST = -c -O2 -D_LARGEFILE64_SOURCE
CFLAGS_HDF5 = -I $(HDF5_PATH)/include
CFLAGS_NCMPI = -I $(NCMPI_PATH)/include
CFLAGS_MPI = -I$(MPI_PATH)/include
#----------------------------------------------------------------------------
# Linker flags
#
# There is a seperate version of the linker flags for each of the _OPT,
# _DEBUG, and _TEST cases.
#----------------------------------------------------------------------------
LFLAGS_OPT = -r8 -i8 -Ur -g -o
LFLAGS_DEBUG = -r8 -i4 -Vaxlib -g -o
LFLAGS_TEST = -r8 -i4 -Vaxlib -o
#----------------------------------------------------------------------------
# Library specific linking
#
# If a FLASH module has a 'LIBRARY xxx' line in its Config file, we need to
# create a macro in this Makefile.h for LIB_xxx, which will be added to the
# link line when FLASH is built. This allows us to switch between different
# (incompatible) libraries. We also create a _OPT, _DEBUG, and _TEST
# library macro to add any performance-minded libraries (like fast math),
# depending on how FLASH was setup.
#----------------------------------------------------------------------------
LIB_HDF4 = -L $(HDF4_PATH)/lib -lmfhdf -ldf -lz -ljpeg
LIB_HDF5 = $(HDF5_PATH)/lib/libhdf5.a -lz
LIB_PNG = -lpng
LIB_MPI =
LIB_NCMPI =
LIB_MPE =
#----------------------------------------------------------------------------
# Additional machine-dependent object files
#
# Add any machine specific files here -- they will be compiled and linked
# when FLASH is built.
#----------------------------------------------------------------------------
MACHOBJ =
#----------------------------------------------------------------------------
# Additional commands
#----------------------------------------------------------------------------
MV = mv -f
AR = ar -r
RM = rm -f
CD = cd
RL = ranlib
ECHO = echo
zeus287{mauche}445: cat Makefile.h
# FLASH makefile definitions for the 64 bit Intel icc/ifort compiler on Linux
# Created for zeus.llnl.gov by nathan and cal.
#
# Fedora Core 3
# Intel Linux Fortran 9.0
# hdf5-1.6.5 (icc90)
# mpich-1.2.7 (icc90)
#----------------------------------------------------------------------------
# Set the HDF/HDF5 library paths -- these need to be updated for your system
#----------------------------------------------------------------------------
MPI_PATH =
HDF4_PATH =
HDF5_PATH =
/usr/global/tools/hdf5/installs/chaos_3_x86_64_ib/hdf5-1.6.5/parallel
ZLIB_PATH =
PAPI_PATH =
PAPI_FLAGS =
NCMPI_PATH =
MPE_PATH =
#----------------------------------------------------------------------------
# Compiler and linker commands
#
# We use the f90 compiler as the linker, so some C libraries may explicitly
# need to be added into the link line.
#----------------------------------------------------------------------------
FCOMP = mpipathf90
CCOMP = mpipathcc
CPPCOMP = mpipathCC
LINK = mpipathf90
# pre-processor flag
PP = -D
#-----------------------------------------------------------------------------
# Compilation flags
#
# Three sets of compilation/linking flags are defined: one for optimized code
# code ("-opt"), one for debugging ("-debug"), and one for testing ("-test").
# Passing these flags to the setup script will cause the value associated with
# the corresponding keys (i.e. those ending in "_OPT", "_DEBUG", or "_TEST") to
# be incorporated into the final Makefile. For example, passing "-opt" to the
# setup script will cause the flags following "FFLAGS_OPT" to be assigned to
# "FFLAGS" in the final Makefile. If none of these flags passed, the default
# behavior will match that of the "-opt" flag.
# In general, "-opt" is meant to optimize compilation and linking. "-debug"
# should enable runtime bounds checking, debugger symbols, and other compiler-
# specific debugging options. "-test" is useful for testing different
# combinations of compiler flags particular to your individual system.
#----------------------------------------------------------------------------
FFLAGS_OPT = -c -r8 -i8 -m64 -march=opteron -O3 -g
FFLAGS_DEBUG = -c -g -r8 -i4 -check bounds -check format -check
output_conversion -warn all -real_size 64
FFLAGS_TEST = -c -r8 -i4 -O2 -real_size 64
CFLAGS_OPT = -c -O3 -r8 -i8 -m64 -march=opteron -g
CFLAGS_DEBUG = -c -g -debug extended -D_LARGEFILE64_SOURCE
CFLAGS_TEST = -c -O2 -D_LARGEFILE64_SOURCE
CFLAGS_HDF5 = -I $(HDF5_PATH)/include
CFLAGS_NCMPI = -I $(NCMPI_PATH)/include
CFLAGS_MPI = -I$(MPI_PATH)/include
#----------------------------------------------------------------------------
# Linker flags
#
# There is a seperate version of the linker flags for each of the _OPT,
# _DEBUG, and _TEST cases.
#----------------------------------------------------------------------------
LFLAGS_OPT = -r8 -i8 -Ur -g -o
LFLAGS_DEBUG = -r8 -i4 -Vaxlib -g -o
LFLAGS_TEST = -r8 -i4 -Vaxlib -o
#----------------------------------------------------------------------------
# Library specific linking
#
# If a FLASH module has a 'LIBRARY xxx' line in its Config file, we need to
# create a macro in this Makefile.h for LIB_xxx, which will be added to the
# link line when FLASH is built. This allows us to switch between different
# (incompatible) libraries. We also create a _OPT, _DEBUG, and _TEST
# library macro to add any performance-minded libraries (like fast math),
# depending on how FLASH was setup.
#----------------------------------------------------------------------------
LIB_HDF4 = -L $(HDF4_PATH)/lib -lmfhdf -ldf -lz -ljpeg
LIB_HDF5 = $(HDF5_PATH)/lib/libhdf5.a -lz
LIB_PNG = -lpng
LIB_MPI =
LIB_NCMPI =
LIB_MPE =
#----------------------------------------------------------------------------
# Additional machine-dependent object files
#
# Add any machine specific files here -- they will be compiled and linked
# when FLASH is built.
#----------------------------------------------------------------------------
MACHOBJ =
#----------------------------------------------------------------------------
# Additional commands
#----------------------------------------------------------------------------
MV = mv -f
AR = ar -r
RM = rm -f
CD = cd
RL = ranlib
ECHO = echo
More information about the flash-users
mailing list