From banerjee at ita.uni-heidelberg.de Fri Feb 1 01:20:29 2008 From: banerjee at ita.uni-heidelberg.de (Robi Banerjee) Date: Fri, 1 Feb 2008 08:20:29 +0100 (CET) Subject: [FLASH-USERS] Question about star formation using FLASH In-Reply-To: References: <47A1D6DB.4070103@astro.rug.nl> <30C8040D-13BB-4956-9E84-F178C2B6C698@flash.uchicago.edu> Message-ID: Dear M.A. we have a simple implementation of 'sink partciles' (ie. particles which are created on the fly and are able to accrete gas) for the FLASH code. We are currently finalizing our work and will publish it soon describing some test cases. If your colleague is willing to wait a while, we can assist him implementing our approach. You can find the outline of our idea in a talk I gave at the FLASH workshop in Bremen: http://www.ita.uni-heidelberg.de/~banerjee/talks/FLASH_Workshop.ppt Cheers, Robi =================== Robi Banerjee Institut f?r Theoretische Astrophysik Universit?t Heidelberg Albert-Ueberle-Str. 2 69120 Heidelberg Germany e-mail: banerjee at ita.uni-heidelberg.de URL : http://www.ita.uni-heidelberg.de/~banerjee tel. : +49 6221 54-8967 fax : +49 6221 544221 On Thu, 31 Jan 2008, Robert Fisher wrote: > > To elaborate upon what John has stated, star particles like those you have > described have not been implemented into FLASH. However, FLASH does support > active, gravitating particles, so the addition of such a feature would be > relatively straightforward -- the algorithm has already been fully laid out, > for instance in Krumholz et al (2004) -- > > http://adsabs.harvard.edu/abs/2004ApJ...611..399K > > I am certain your colleague could obtain assistance where needed from both > the FLASH code group and the user community. > > Best wishes, > > Bob > > On Thu, 31 Jan 2008, John ZuHone wrote: > >> Hello M.A., >> >> Such capabilities are not currently in FLASH2 and they will not be in >> the first release of FLASH3, though they *may* be in future releases. >> Others have managed to get this working for their own use. >> >> Strictly speaking one would not be converting gas particles (since >> we're working with Eulerian grid variables) but would be creating massive >> star particles from scratch at a particular grid location once conditions >> were met at that location (say, density rises to a certain value and >> temperature falls below a certain value). Then the star particle would be >> taking its velocity as well from the local velocity on the grid. >> >> In essence, this is not currently implemented but once one had a >> prescription for this it should not be difficult to implement. >> >> Best, >> >> John Z >> >> On Jan 31, 2008, at 8:10 AM, M.A. Latife wrote: >> >>> >>> Hi all, >>> one of my colleagues is interested in using FLASH, His question is as >>> follows, i Hope you people can reply his question in better way. >>> "I want to simulate star formation starting with gas particles. Is Flash >>> able to convert gas particles (when a certain condition is met) into star >>> particles? This because I want to see the initial mass function. Do I have >>> to write my own module for this, or is there such an option already there? >>> " >>> >>> Thanks, >>> M.A.Latife > From seyit at astro.rug.nl Fri Feb 1 02:31:41 2008 From: seyit at astro.rug.nl (Seyit Hocuk) Date: Fri, 01 Feb 2008 09:31:41 +0100 Subject: [FLASH-USERS] Question about star formation using FLASH In-Reply-To: References: <47A1D6DB.4070103@astro.rug.nl> <30C8040D-13BB-4956-9E84-F178C2B6C698@flash.uchicago.edu> Message-ID: <47A2D8ED.8030801@astro.rug.nl> Hi Everybody, I am this colleague everyone speaks of. As of today I am a listed to the flash-users and can reply myself. First of All, thanks to everyone who has answered in such a short notice and have helped me very much. It's very appreciated. I will now sink my teeth into Flash and try to learn it. Robi: I am glad that you are working on this and yes I am willing to wait a while. It would be great if I could implement your work indeed. Bob: Thanks for the usefull link John Z: Thanks Cheers, Seyit Robi Banerjee wrote: > Dear M.A. > > we have a simple implementation of 'sink partciles' (ie. particles > which are created on the fly and are able to accrete gas) for the > FLASH code. We are currently finalizing our work and will publish it > soon describing some test cases. > > If your colleague is willing to wait a while, we can assist him > implementing our approach. You can find the outline of our idea > in a talk I gave at the FLASH workshop in Bremen: > http://www.ita.uni-heidelberg.de/~banerjee/talks/FLASH_Workshop.ppt > > Cheers, > Robi > > ====================================== > Robi Banerjee > Institut f?r Theoretische Astrophysik > Universit?t Heidelberg > Albert-Ueberle-Str. 2 > 69120 Heidelberg > Germany > e-mail: banerjee at ita.uni-heidelberg.de > URL : http://www.ita.uni-heidelberg.de/~banerjee > tel. : +49 6221 54-8967 > fax : +49 6221 544221 > > On Thu, 31 Jan 2008, Robert Fisher wrote: > >> >> To elaborate upon what John has stated, star particles like those you >> have described have not been implemented into FLASH. However, FLASH >> does support active, gravitating particles, so the addition of such a >> feature would be relatively straightforward -- the algorithm has >> already been fully laid out, for instance in Krumholz et al (2004) -- >> >> http://adsabs.harvard.edu/abs/2004ApJ...611..399K >> >> I am certain your colleague could obtain assistance where needed from >> both the FLASH code group and the user community. >> >> Best wishes, >> >> Bob >> >> On Thu, 31 Jan 2008, John ZuHone wrote: >> >>> Hello M.A., >>> >>> Such capabilities are not currently in FLASH2 and they will not >>> be in the first release of FLASH3, though they *may* be in future >>> releases. Others have managed to get this working for their own use. >>> >>> Strictly speaking one would not be converting gas particles >>> (since we're working with Eulerian grid variables) but would be >>> creating massive star particles from scratch at a particular grid >>> location once conditions were met at that location (say, density >>> rises to a certain value and temperature falls below a certain >>> value). Then the star particle would be taking its velocity as well >>> from the local velocity on the grid. >>> >>> In essence, this is not currently implemented but once one had a >>> prescription for this it should not be difficult to implement. >>> >>> Best, >>> >>> John Z >>> >>> On Jan 31, 2008, at 8:10 AM, M.A. Latife wrote: >>> >>>> >>>> Hi all, >>>> one of my colleagues is interested in using FLASH, His question is >>>> as follows, i Hope you people can reply his question in better way. >>>> "I want to simulate star formation starting with gas particles. Is >>>> Flash able to convert gas particles (when a certain condition is >>>> met) into star particles? This because I want to see the initial >>>> mass function. Do I have to write my own module for this, or is >>>> there such an option already there? " >>>> >>>> Thanks, >>>> M.A.Latife >> From seyit at astro.rug.nl Wed Feb 6 07:48:46 2008 From: seyit at astro.rug.nl (Seyit Hocuk) Date: Wed, 06 Feb 2008 14:48:46 +0100 Subject: [FLASH-USERS] Plotting Flash Data Message-ID: <47A9BABE.5010204@astro.rug.nl> Hi, I wanted to ask you how you view the results from Flash. What method do you use to plot these HDF5 files? I tried VISIT, but I am very limited in plotting 2D plots with this program. So now I try Python/Matlab, however many problems arise. I just want to plot Density versus Radius at this moment. Thanks, Seyit From hudson at mcs.anl.gov Wed Feb 6 09:14:01 2008 From: hudson at mcs.anl.gov (Randy Hudson) Date: Wed, 06 Feb 2008 09:14:01 -0600 Subject: [FLASH-USERS] Plotting Flash Data In-Reply-To: <47A9BABE.5010204@astro.rug.nl> References: <47A9BABE.5010204@astro.rug.nl> Message-ID: <47A9CEB9.5040309@mcs.anl.gov> Seyit, If you mean that you want to plot a graph of density against its location along a radius, I think Lineout, in VisIt will do. I'm trying that out now. Seyit Hocuk wrote: > Hi, > > I wanted to ask you how you view the results from Flash. What method do > you use to plot these HDF5 files? > I tried VISIT, but I am very limited in plotting 2D plots with this > program. So now I try Python/Matlab, however many problems arise. > I just want to plot Density versus Radius at this moment. > > Thanks, > Seyit > > -- Randy. From seyit at astro.rug.nl Wed Feb 6 09:35:11 2008 From: seyit at astro.rug.nl (Seyit Hocuk) Date: Wed, 06 Feb 2008 16:35:11 +0100 Subject: [FLASH-USERS] Plotting Flash Data In-Reply-To: <47A9CEB9.5040309@mcs.anl.gov> References: <47A9BABE.5010204@astro.rug.nl> <47A9CEB9.5040309@mcs.anl.gov> Message-ID: <47A9D3AF.9090402@astro.rug.nl> Thanks Randy, Unfortunately, with my data (Sedov testproblem) there is no option for me to select Lineout, nor curve plot for that matter. Let me know if you find another way. Cheers, Seyit Randy Hudson wrote: > > Seyit, > > If you mean that you want to plot a graph of density against its > location along a radius, I think Lineout, in VisIt will do. I'm > trying that out now. > > From hudson at mcs.anl.gov Wed Feb 6 09:57:22 2008 From: hudson at mcs.anl.gov (Randy Hudson) Date: Wed, 06 Feb 2008 09:57:22 -0600 Subject: [FLASH-USERS] Plotting Flash Data In-Reply-To: <47A9BABE.5010204@astro.rug.nl> References: <47A9BABE.5010204@astro.rug.nl> Message-ID: <47A9D8E2.3060806@mcs.anl.gov> Seyit, Attached is a screen snap of 2 visit display windows showing a "Pseudocolor" plot (on the left) of a 2d flash dataset, and the curve (in green; ignore the red) created by a "Lineout" sampling of density along a radius. The sampling line can be seen in the pseudocolor plot. I produced this by doing the following: o read the file o display a "Pseudocolor" plot of the file o open the "Query" window from the "Controls" menu o in the query window o scroll down in the list of queries to "Lineout" and select it o enter the start and end coordinates of the sample line o leave "Variables" as "default" o click on "Query" A new display window will be opened for the curve. Let me know if that's not what you needed. Seyit Hocuk wrote: > Hi, > > I wanted to ask you how you view the results from Flash. What method do > you use to plot these HDF5 files? > I tried VISIT, but I am very limited in plotting 2D plots with this > program. So now I try Python/Matlab, however many problems arise. > I just want to plot Density versus Radius at this moment. > > Thanks, > Seyit > > -- Randy. -------------- next part -------------- A non-text attachment was scrubbed... Name: Picture 1.png Type: image/png Size: 190854 bytes Desc: not available Url : http://flash.uchicago.edu/pipermail/flash-users/attachments/20080206/a9e47821/attachment-0001.png From latife at astro.rug.nl Thu Feb 7 07:08:47 2008 From: latife at astro.rug.nl (M.A. Latife) Date: Thu, 07 Feb 2008 14:08:47 +0100 Subject: [FLASH-USERS] how to Implement Isothermal EOS Message-ID: <47AB02DF.2020603@astro.rug.nl> Hi all, I am using FLASH2.5. Is there any way to implement isothermal EOS. should i use heating and cooling source terms for this purpose or is there any other way to do this? kind Regards M.A.Latife From tomek at scs.fsu.edu Thu Feb 7 07:47:30 2008 From: tomek at scs.fsu.edu (Tomasz Plewa) Date: Thu, 07 Feb 2008 08:47:30 -0500 Subject: [FLASH-USERS] how to Implement Isothermal EOS In-Reply-To: <47AB02DF.2020603@astro.rug.nl> References: <47AB02DF.2020603@astro.rug.nl> Message-ID: <20080207084730.mxwn5rmey8ocgg4g@webmail.scs.fsu.edu> M.A. - You may consult Balsara, D., 1994, ApJ, 420, 197. There is also more recent paper by Andrea Mignone (2007, JCP, 225, 1427). Some other codes do have an isothermal EoS implemented (CLAW, Athena, VH-1) as well as groups working on the evolution of protoplanetary disks. I seem to remember a paper (Blondin? Stone?) giving the exact formulae, but I don't have it handy. The brute-force approach is to use a constant gamma-law (ideal) gas EoS with gamma as close to 1.e0 as possible (e.g. 1.001). But in this case the solution may be numerically unstable and some provisions needs to be build into the Riemann solver. Using (naively) gamma=1 will almost certainly break the code. The bottom line is you need to replace the existing Riemann solver with the isothermal version and implement an isothermal EoS. Hope this helps. Tomek -- School of Computational Science, DSL 443 ph: 850.644.1959 Florida State University fax: 850.644.0098 Tallahassee, FL 32306 web: people.scs.fsu.edu/~tomek/ -- M.A. Latife wrote: > Hi all, > I am using FLASH2.5. Is there any way to implement isothermal EOS. > should i use heating and cooling source terms for this purpose or is > there any other way to do this? > kind Regards > M.A.Latife > > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From banerjee at ita.uni-heidelberg.de Thu Feb 7 08:11:30 2008 From: banerjee at ita.uni-heidelberg.de (Robi Banerjee) Date: Thu, 7 Feb 2008 15:11:30 +0100 (CET) Subject: [FLASH-USERS] how to Implement Isothermal EOS In-Reply-To: <47AB02DF.2020603@astro.rug.nl> References: <47AB02DF.2020603@astro.rug.nl> Message-ID: Hi M.A. we used the cooling module to implement an isothermal EOS. It worked fine for most of our cases although we did not modify the Riemann solver. Cheers, Robi On Thu, 7 Feb 2008, M.A. Latife wrote: > Hi all, > I am using FLASH2.5. Is there any way to implement isothermal EOS. should i > use heating and cooling source terms for this purpose or is there any other > way to do this? > kind Regards > M.A.Latife > > From Kevin.M.Olson at drexel.edu Thu Feb 7 08:18:30 2008 From: Kevin.M.Olson at drexel.edu (Kevin Olson) Date: Thu, 7 Feb 2008 09:18:30 -0500 Subject: [FLASH-USERS] how to Implement Isothermal EOS In-Reply-To: <47AB02DF.2020603@astro.rug.nl> References: <47AB02DF.2020603@astro.rug.nl> Message-ID: <0bfc123d42b3bbfa9f16f4a10b96cfa0@drexel.edu> I have implemented both things Tomasz suggested and they seem to work fine. The easiest is to set gamma = 1.01 or so (too much smaller and things are unstable) or to simply add a cooling/heating term that keeps the temperature fixed. The riemann solver does not need to be modified for either of these approaches since you are still using the same EOS. Kevin Olson On Feb 7, 2008, at 8:08 AM, M.A. Latife wrote: > Hi all, > I am using FLASH2.5. Is there any way to implement isothermal EOS. > should i use heating and cooling source terms for this purpose or is > there any other way to do this? > kind Regards > M.A.Latife > From rfisher at flash.uchicago.edu Thu Feb 7 08:34:43 2008 From: rfisher at flash.uchicago.edu (Robert Fisher) Date: Thu, 7 Feb 2008 08:34:43 -0600 (CST) Subject: [FLASH-USERS] how to Implement Isothermal EOS In-Reply-To: <0bfc123d42b3bbfa9f16f4a10b96cfa0@drexel.edu> References: <47AB02DF.2020603@astro.rug.nl> <0bfc123d42b3bbfa9f16f4a10b96cfa0@drexel.edu> Message-ID: To add to what others have said, the "adiabatic" approximation of setting gamma to a small value is a commonly-used one which should work well. There is, however, a very substantial caveat. The key constraint is that because the gas still remains weakly adiabatic, compressional heating will eventually cause the flow to deviate substantially from isothermality. What one needs is to identify the peak compression for a given problem, and to set the gamma small enough such that the gas will remain isothermal through that compression. As Kevin says, running with small deviations from 1 should not require any modifications. It is possible to run with values much closer to unity with some minor modifications to the Riemann solver as Tomek has also indicated. Best wishes, Bob On Thu, 7 Feb 2008, Kevin Olson wrote: > I have implemented both things Tomasz suggested and they seem to work fine. > The easiest is to set gamma = 1.01 or so (too much smaller and things are > unstable) > or to simply add a cooling/heating term that keeps the temperature fixed. > The riemann solver does not need to be modified for either of these > approaches since you are still using the same EOS. > > Kevin Olson > > On Feb 7, 2008, at 8:08 AM, M.A. Latife wrote: > >> Hi all, >> I am using FLASH2.5. Is there any way to implement isothermal EOS. should i >> use heating and cooling source terms for this purpose or is there any other >> way to do this? >> kind Regards >> M.A.Latife >> > From mateuszr at umich.edu Mon Feb 11 12:53:26 2008 From: mateuszr at umich.edu (mateuszr at umich.edu) Date: Mon, 11 Feb 2008 13:53:26 -0500 Subject: [FLASH-USERS] Sedov scaling test Message-ID: <20080211135326.exjfnl9eb08w08g4@web.mail.umich.edu> Hi all, I am testing a very small Dell cluster (8 cpus total). The machine is a dual quad core Xeon with 2 GB per core. The OS is Rocks 4.3, and the compilers used is the stock gcc 3.4.6 with mpich 1.2.7, mpif90 (Makefile.h enclosed below). FLASH version is 2.5 and I am using a Sedov test setup. The setup does not seem to scale with the number of processors at all. If anything, the execution time increases very slightly as I double the number of processors. I also noticed that all timesteps are printed out to the screen n times, where n is the number of processors. It seems to me that for some reason, the code just runs the same calculation n times instead of distributing the load evenly among all processors. It seems to me that this problem has nothing to do with the processor communication and, quite likely, is (embarrasingly?) trivial to solve. Has anybody seen this behavior before ? thanks, Mateusz P.S. I am enclosing Makefile.h below: # FLASH makefile definitions for ix86 Linux (Portland Group compiler) # note, in order to get pgprof to work properly, it was necessary to # download a new version from ftp://ftp.pgroup.com/ that plays nicely # with GNOME. #---------------------------------------------------------------------------- # Set the HDF/HDF5 library paths -- these need to be updated for your system #---------------------------------------------------------------------------- #HDF4_PATH = /usr/local/hdf4 #HDF5_PATH = /opt/HDF5-1.4.2-patch1-serial/ HDF5_PATH = /opt/packages/hdf5-1.6.6/ ZLIB_PATH = PAPI_PATH = PAPI_FLAGS = NCMPI_PATH = MPE_PATH = #---------------------------------------------------------------------------- # Compiler and linker commands # # Use the MPICH wrappers around the compilers -- these will automatically # load the proper libraries and include files. Version of MPICH prior # to 1.2.2 (?) do not recognize .F90 as a valid Fortran file extension. # You need to edit mpif90 and add .F90 to the test of filename extensions, # or upgrade your MPICH. #---------------------------------------------------------------------------- FCOMP = /opt/mpich/intel/bin/mpif90 CCOMP = /opt/mpich/intel/bin/mpicc CPPCOMP = /opt/mpich/intel/bin/mpicc LINK = /opt/mpich/intel/bin/mpif90 # pre-processor flag PP = -D #---------------------------------------------------------------------------- # Compilation flags # # Three sets of compilation/linking flags are defined: one for optimized # code, one for testing, and one for debugging. The default is to use the # _OPT version. Specifying -debug to setup will pick the _DEBUG version, # these should enable bounds checking. Specifying _TEST is used for # flash_test, and is set for quick code generation, and (sometimes) # profiling. The Makefile generated by setup will assign the generic token # (ex. FFLAGS) to the proper set of flags (ex. FFLAGS_OPT). #---------------------------------------------------------------------------- #FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/packages/hdf5-1.6.6/include -I/opt/mpich/intel/include/ -L/opt/mpich/intel/lib -L/opt/packages/hdf5-1.6.6/lib -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib -lhdf5 -lrfftw_mpi -lfftw_mpi -lrfftw FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/packages/hdf5-1.6.6/include -I/opt/mpich/intel/include/ -L/opt/mpich/intel/lib -L/opt/packages/hdf5-1.6.6/lib -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib -ldrfftw_mpi -ldfftw_mpi -ldrfftw -ldfftw # FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/usr/include/openmpi -I/usr/include/openmpi/64/ -L/usr/lib/openmpi #FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/mpich/gnu/include -L/opt/mpich/gnu/lib FFLAGS_DEBUG = -g -c -Mbounds -r8 -i4 FFLAGS_TEST = -c -r8 -i4 -Mprof=lines F90FLAGS = #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include -I/usr/include/openmpi -I/usr/include/openmpi/64/ -L/usr/lib/openmpi #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include -I/opt/mpich/gnu/include -L/opt/mpich/gnu/lib #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include -I/opt/intel/impi/3.1/include64/ -L/opt/intel/impi/3.1/lib64/ -L/opt/packages/hdf5-1.6.6/lib -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib -lhdf5 -lrfftw_mpi -lfftw_mpi -lrttfw CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include -I/opt/intel/impi/3.1/include64/ -L/opt/intel/impi/3.1/lib64/ -L/opt/packages/hdf5-1.6.6/lib -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib -lhdf5 -ldrfftw_mpi -ldfftw_mpi -ldrfftw -ldfftw CFLAGS_DEBUG = -g -c CFLAGS_TEST = -c # if we are using HDF5, we need to specify the path to the include files CFLAGS_HDF5 = -I $(HDF5_PATH)/include CFLAGS_NCMPI = #---------------------------------------------------------------------------- # Linker flags # # There is a seperate version of the linker flags for each of the _OPT, # _DEBUG, and _TEST cases. #---------------------------------------------------------------------------- LFLAGS_OPT = -o LFLAGS_DEBUG = -g -o LFLAGS_TEST = -Mprof=lines -o #---------------------------------------------------------------------------- # Library specific linking # # If a FLASH module has a 'LIBRARY xxx' line in its Config file, we need to # create a macro in this Makefile.h for LIB_xxx, which will be added to the # link line when FLASH is built. This allows us to switch between different # (incompatible) libraries. We also create a _OPT, _DEBUG, and _TEST # library macro to add any performance-minded libraries (like fast math), # depending on how FLASH was setup. #---------------------------------------------------------------------------- LIB_OPT = LIB_DEBUG = LIB_TEST = #LIB_MATH = -L/u/mateusz/fftw-2.1.5/lib -ldrfftw_mpi -ldfftw_mpi -ldrfftw -ldfftw #LIB_MATH = -L/opt/packages/fftw-2.1.5/lib -ldrfftw_mpi -ldfftw_mpi -ldrfftw -ldfftw LIB_MATH = -L/opt/packages/fftw-2.1.5/lib -L/opt/packages/hdf5-1.6.6/lib -lrfftw_mpi -lfftw_mpi -lrfftw -lfftw -lhdf5 #LIB_HDF4 = -L $(HDF4_PATH)/lib -lmfhdf -ldf -ljpeg -lz LIB_HDF5 = -L $(HDF5_PATH)/lib $(LIB_MATH) -lmpi -lz -lsz #LIB_HDF5 = -lhdf5 $(LIB_MATH) -lmpi LIB_PAPI = LIB_NCMPI = LIB_MPE = #---------------------------------------------------------------------------- # Additional machine-dependent object files # # Add any machine specific files here -- they will be compiled and linked # when FLASH is built. #---------------------------------------------------------------------------- MACHOBJ = #---------------------------------------------------------------------------- # Additional commands #---------------------------------------------------------------------------- MV = mv -f AR = ar -r RM = rm -f CD = cd RL = ranlib ECHO = echo From chan at mcs.anl.gov Mon Feb 11 13:33:42 2008 From: chan at mcs.anl.gov (Anthony Chan) Date: Mon, 11 Feb 2008 13:33:42 -0600 (CST) Subject: [FLASH-USERS] Sedov scaling test In-Reply-To: <20080211135326.exjfnl9eb08w08g4@web.mail.umich.edu> References: <20080211135326.exjfnl9eb08w08g4@web.mail.umich.edu> Message-ID: On Mon, 11 Feb 2008, mateuszr at umich.edu wrote: > > Hi all, > > I am testing a very small Dell cluster (8 cpus total). The machine is a dual > quad core Xeon with 2 GB per core. The OS is Rocks 4.3, and the compilers > used is the stock gcc 3.4.6 with mpich 1.2.7, mpif90 (Makefile.h enclosed > below). FLASH version is 2.5 and I am using a Sedov test setup. > > The setup does not seem to scale with the number of processors at all. > If anything, the execution time increases very slightly as I double the > number of processors. I also noticed that all timesteps are printed out to > the screen n times, where n is the number of processors. It seems to me that > for some reason, the code just runs the same calculation n times instead of > distributing the load > evenly among all processors. It is posssible that you have n MPI_COMM_WORLD where each has only 1 process (instead of 1 MPI_COMM_WORLD with n processes). You should use MPICH2 which is more robust than MPICH1 especially in process management. A.Chan > > It seems to me that this problem has nothing to do with the processor > communication and, quite likely, is (embarrasingly?) trivial to solve. > > Has anybody seen this behavior before ? > > thanks, > Mateusz > > > P.S. I am enclosing Makefile.h below: > > # FLASH makefile definitions for ix86 Linux (Portland Group compiler) > > # note, in order to get pgprof to work properly, it was necessary to # > download a new > version from ftp://ftp.pgroup.com/ that plays nicely > # with GNOME. > > #---------------------------------------------------------------------------- > # Set the HDF/HDF5 library paths -- these need to be updated for your system > #---------------------------------------------------------------------------- > #HDF4_PATH = /usr/local/hdf4 > #HDF5_PATH = /opt/HDF5-1.4.2-patch1-serial/ > HDF5_PATH = /opt/packages/hdf5-1.6.6/ > > ZLIB_PATH = > > PAPI_PATH = > PAPI_FLAGS = > > NCMPI_PATH = > MPE_PATH = > > #---------------------------------------------------------------------------- > # Compiler and linker commands > # > # Use the MPICH wrappers around the compilers -- these will automatically > # load the proper libraries and include files. Version of MPICH prior > # to 1.2.2 (?) do not recognize .F90 as a valid Fortran file extension. > # You need to edit mpif90 and add .F90 to the test of filename extensions, > # or upgrade your MPICH. > #---------------------------------------------------------------------------- > FCOMP = /opt/mpich/intel/bin/mpif90 > CCOMP = /opt/mpich/intel/bin/mpicc > CPPCOMP = /opt/mpich/intel/bin/mpicc > LINK = /opt/mpich/intel/bin/mpif90 > > # pre-processor flag > PP = -D > > #---------------------------------------------------------------------------- > # Compilation flags > # > # Three sets of compilation/linking flags are defined: one for optimized > # code, one for testing, and one for debugging. The default is to use the # > _OPT > version. Specifying -debug to setup will pick the _DEBUG version, > # these should enable bounds checking. Specifying _TEST is used for # > flash_test, and > is set for quick code generation, and (sometimes) # profiling. The Makefile > generated > by setup will assign the generic token # (ex. FFLAGS) to the proper set of > flags (ex. > FFLAGS_OPT). > #---------------------------------------------------------------------------- > #FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/packages/hdf5-1.6.6/include > -I/opt/mpich/intel/include/ -L/opt/mpich/intel/lib > -L/opt/packages/hdf5-1.6.6/lib > -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib -lhdf5 > -lrfftw_mpi > -lfftw_mpi -lrfftw > FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/packages/hdf5-1.6.6/include > -I/opt/mpich/intel/include/ -L/opt/mpich/intel/lib > -L/opt/packages/hdf5-1.6.6/lib > -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib > -ldrfftw_mpi > -ldfftw_mpi -ldrfftw -ldfftw > # FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/usr/include/openmpi > -I/usr/include/openmpi/64/ -L/usr/lib/openmpi > #FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/mpich/gnu/include > -L/opt/mpich/gnu/lib > FFLAGS_DEBUG = -g -c -Mbounds -r8 -i4 FFLAGS_TEST = -c -r8 -i4 -Mprof=lines > > F90FLAGS = > > #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include > -I/usr/include/openmpi > -I/usr/include/openmpi/64/ -L/usr/lib/openmpi > #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include > -I/opt/mpich/gnu/include > -L/opt/mpich/gnu/lib > #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include > -I/opt/intel/impi/3.1/include64/ > -L/opt/intel/impi/3.1/lib64/ -L/opt/packages/hdf5-1.6.6/lib > -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib -lhdf5 > -lrfftw_mpi > -lfftw_mpi -lrttfw > CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include > -I/opt/intel/impi/3.1/include64/ > -L/opt/intel/impi/3.1/lib64/ -L/opt/packages/hdf5-1.6.6/lib > -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib -lhdf5 > -ldrfftw_mpi > -ldfftw_mpi -ldrfftw -ldfftw > > CFLAGS_DEBUG = -g -c > CFLAGS_TEST = -c > > # if we are using HDF5, we need to specify the path to the include files > CFLAGS_HDF5 = -I $(HDF5_PATH)/include > > CFLAGS_NCMPI = > > #---------------------------------------------------------------------------- > # Linker flags > # > # There is a seperate version of the linker flags for each of the _OPT, # > _DEBUG, and > _TEST cases. > #---------------------------------------------------------------------------- > > LFLAGS_OPT = -o > LFLAGS_DEBUG = -g -o > LFLAGS_TEST = -Mprof=lines -o > > > #---------------------------------------------------------------------------- > # Library specific linking > # > # If a FLASH module has a 'LIBRARY xxx' line in its Config file, we need to > # create a macro in this Makefile.h for LIB_xxx, which will be added to the > # link line when FLASH is built. This allows us to switch between different > # (incompatible) libraries. We also create a _OPT, _DEBUG, and _TEST > # library macro to add any performance-minded libraries (like fast math), > # depending on how FLASH was setup. > #---------------------------------------------------------------------------- > > LIB_OPT = LIB_DEBUG = > LIB_TEST = > > > #LIB_MATH = -L/u/mateusz/fftw-2.1.5/lib -ldrfftw_mpi -ldfftw_mpi -ldrfftw > -ldfftw > #LIB_MATH = -L/opt/packages/fftw-2.1.5/lib -ldrfftw_mpi -ldfftw_mpi -ldrfftw > -ldfftw > LIB_MATH = -L/opt/packages/fftw-2.1.5/lib -L/opt/packages/hdf5-1.6.6/lib > -lrfftw_mpi > -lfftw_mpi -lrfftw -lfftw -lhdf5 #LIB_HDF4 = -L $(HDF4_PATH)/lib -lmfhdf > -ldf -ljpeg -lz > LIB_HDF5 = -L $(HDF5_PATH)/lib $(LIB_MATH) -lmpi -lz -lsz > #LIB_HDF5 = -lhdf5 $(LIB_MATH) -lmpi > > LIB_PAPI = > > LIB_NCMPI = > LIB_MPE = > > #---------------------------------------------------------------------------- > # Additional machine-dependent object files > # > # Add any machine specific files here -- they will be compiled and linked > # when FLASH is built. > #---------------------------------------------------------------------------- > > MACHOBJ = > > > #---------------------------------------------------------------------------- > # Additional commands > #---------------------------------------------------------------------------- > > MV = mv -f > AR = ar -r > RM = rm -f > CD = cd > RL = ranlib > ECHO = echo > > > From cyoung at grad.physics.sunysb.edu Mon Feb 11 13:49:53 2008 From: cyoung at grad.physics.sunysb.edu (Clint Young) Date: Mon, 11 Feb 2008 14:49:53 -0500 (EST) Subject: [FLASH-USERS] Sedov scaling test In-Reply-To: <20080211135326.exjfnl9eb08w08g4@web.mail.umich.edu> Message-ID: On Mon, 11 Feb 2008 mateuszr at umich.edu wrote: > > Hi all, > > I am testing a very small Dell cluster (8 cpus total). The machine is a > dual quad core Xeon with 2 GB per core. The OS is Rocks 4.3, and the > compilers used is the stock gcc 3.4.6 with mpich 1.2.7, mpif90 > (Makefile.h enclosed below). FLASH version is 2.5 and I am using a > Sedov test setup. > > The setup does not seem to scale with the number of processors at all. > If anything, the execution time increases very slightly as I double the > number of processors. I also noticed that all timesteps are printed out > to the screen n times, where n is the number of processors. It seems to > me that for some reason, the code just runs the same calculation n > times instead of distributing the load > evenly among all processors. > > It seems to me that this problem has nothing to do with the processor > communication and, quite likely, is (embarrasingly?) trivial to solve. > > Has anybody seen this behavior before ? > > thanks, > Mateusz > > > P.S. I am enclosing Makefile.h below: > > # FLASH makefile definitions for ix86 Linux (Portland Group compiler) > > # note, in order to get pgprof to work properly, it was necessary to # > download a new > version from ftp://ftp.pgroup.com/ that plays nicely > # with GNOME. > > #---------------------------------------------------------------------------- > # Set the HDF/HDF5 library paths -- these need to be updated for your system > #---------------------------------------------------------------------------- > #HDF4_PATH = /usr/local/hdf4 > #HDF5_PATH = /opt/HDF5-1.4.2-patch1-serial/ > HDF5_PATH = /opt/packages/hdf5-1.6.6/ > > ZLIB_PATH = > > PAPI_PATH = > PAPI_FLAGS = > > NCMPI_PATH = > MPE_PATH = > > #---------------------------------------------------------------------------- > # Compiler and linker commands > # > # Use the MPICH wrappers around the compilers -- these will automatically > # load the proper libraries and include files. Version of MPICH prior > # to 1.2.2 (?) do not recognize .F90 as a valid Fortran file extension. > # You need to edit mpif90 and add .F90 to the test of filename extensions, > # or upgrade your MPICH. > #---------------------------------------------------------------------------- > FCOMP = /opt/mpich/intel/bin/mpif90 > CCOMP = /opt/mpich/intel/bin/mpicc > CPPCOMP = /opt/mpich/intel/bin/mpicc > LINK = /opt/mpich/intel/bin/mpif90 > > # pre-processor flag > PP = -D > > #---------------------------------------------------------------------------- > # Compilation flags > # > # Three sets of compilation/linking flags are defined: one for optimized > # code, one for testing, and one for debugging. The default is to use > the # _OPT > version. Specifying -debug to setup will pick the _DEBUG version, > # these should enable bounds checking. Specifying _TEST is used for # > flash_test, and > is set for quick code generation, and (sometimes) # profiling. The > Makefile generated > by setup will assign the generic token # (ex. FFLAGS) to the proper > set of flags (ex. > FFLAGS_OPT). > #---------------------------------------------------------------------------- > #FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/packages/hdf5-1.6.6/include > -I/opt/mpich/intel/include/ -L/opt/mpich/intel/lib > -L/opt/packages/hdf5-1.6.6/lib > -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib > -lhdf5 -lrfftw_mpi > -lfftw_mpi -lrfftw > FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/packages/hdf5-1.6.6/include > -I/opt/mpich/intel/include/ -L/opt/mpich/intel/lib > -L/opt/packages/hdf5-1.6.6/lib > -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib -ldrfftw_mpi > -ldfftw_mpi -ldrfftw -ldfftw > # FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/usr/include/openmpi > -I/usr/include/openmpi/64/ -L/usr/lib/openmpi > #FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/mpich/gnu/include > -L/opt/mpich/gnu/lib > FFLAGS_DEBUG = -g -c -Mbounds -r8 -i4 FFLAGS_TEST = -c -r8 -i4 -Mprof=lines > > F90FLAGS = > > #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include > -I/usr/include/openmpi > -I/usr/include/openmpi/64/ -L/usr/lib/openmpi > #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include > -I/opt/mpich/gnu/include > -L/opt/mpich/gnu/lib > #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include > -I/opt/intel/impi/3.1/include64/ > -L/opt/intel/impi/3.1/lib64/ -L/opt/packages/hdf5-1.6.6/lib > -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib > -lhdf5 -lrfftw_mpi > -lfftw_mpi -lrttfw > CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include > -I/opt/intel/impi/3.1/include64/ > -L/opt/intel/impi/3.1/lib64/ -L/opt/packages/hdf5-1.6.6/lib > -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib > -lhdf5 -ldrfftw_mpi > -ldfftw_mpi -ldrfftw -ldfftw > > CFLAGS_DEBUG = -g -c > CFLAGS_TEST = -c > > # if we are using HDF5, we need to specify the path to the include files > CFLAGS_HDF5 = -I $(HDF5_PATH)/include > > CFLAGS_NCMPI = > > #---------------------------------------------------------------------------- > # Linker flags > # > # There is a seperate version of the linker flags for each of the > _OPT, # _DEBUG, and > _TEST cases. > #---------------------------------------------------------------------------- > > LFLAGS_OPT = -o > LFLAGS_DEBUG = -g -o > LFLAGS_TEST = -Mprof=lines -o > > > #---------------------------------------------------------------------------- > # Library specific linking > # > # If a FLASH module has a 'LIBRARY xxx' line in its Config file, we need to > # create a macro in this Makefile.h for LIB_xxx, which will be added to the > # link line when FLASH is built. This allows us to switch between different > # (incompatible) libraries. We also create a _OPT, _DEBUG, and _TEST > # library macro to add any performance-minded libraries (like fast math), > # depending on how FLASH was setup. > #---------------------------------------------------------------------------- > > LIB_OPT = LIB_DEBUG = > LIB_TEST = > > > #LIB_MATH = -L/u/mateusz/fftw-2.1.5/lib -ldrfftw_mpi -ldfftw_mpi > -ldrfftw -ldfftw > #LIB_MATH = -L/opt/packages/fftw-2.1.5/lib -ldrfftw_mpi -ldfftw_mpi > -ldrfftw -ldfftw > LIB_MATH = -L/opt/packages/fftw-2.1.5/lib > -L/opt/packages/hdf5-1.6.6/lib -lrfftw_mpi > -lfftw_mpi -lrfftw -lfftw -lhdf5 #LIB_HDF4 = -L $(HDF4_PATH)/lib > -lmfhdf -ldf -ljpeg -lz > LIB_HDF5 = -L $(HDF5_PATH)/lib $(LIB_MATH) -lmpi -lz -lsz > #LIB_HDF5 = -lhdf5 $(LIB_MATH) -lmpi > > LIB_PAPI = > > LIB_NCMPI = > LIB_MPE = > > #---------------------------------------------------------------------------- > # Additional machine-dependent object files > # > # Add any machine specific files here -- they will be compiled and linked > # when FLASH is built. > #---------------------------------------------------------------------------- > > MACHOBJ = > > > #---------------------------------------------------------------------------- > # Additional commands > #---------------------------------------------------------------------------- > > MV = mv -f > AR = ar -r > RM = rm -f > CD = cd > RL = ranlib > ECHO = echo > > From gawrysz at camk.edu.pl Tue Feb 12 00:29:43 2008 From: gawrysz at camk.edu.pl (Artur Gawryszczak) Date: Tue, 12 Feb 2008 07:29:43 +0100 Subject: [FLASH-USERS] Sedov scaling test In-Reply-To: <20080211135326.exjfnl9eb08w08g4@web.mail.umich.edu> References: <20080211135326.exjfnl9eb08w08g4@web.mail.umich.edu> Message-ID: <200802120729.43471@phoenix.camk.edu.pl> Hi Mateusz, On poniedzia?ek, 11 lutego 2008, mateuszr at umich.edu wrote: > I am testing a very small Dell cluster (8 cpus total). The machine is a > dual quad core Xeon with 2 GB per core. [...] The setup does not seem to > scale with the number of processors at all. I noticed similar behavior. There was significant speedup when I've changed from one to two cores (other were idle), there was some speedup when I tried four cores and almost nothing when I tried all eight. The same setup (code and parameters) did scale well up to 8 processes on a cluster of dual Opterons with Infiniband interconnects despite of usage of selfgravity module (lots of communication, scaling significantly more difficult than in hydro). I tried to pin threads to certain cores (taskset/schedtool) and discovered that the problem with scaling is most likely due to insufficient bandwith between CPU cores and main memory. All four cores of a Quad Core Xeon share single memory bus and when two or more cores of the same physical CPU needs to read or write main memory only one can do the transfer and others have to wait. This is a bottleneck of Intel systems and in general might be a problem of other multi-core CPUs. If Dual Quad Core Intel system has NUMA architecture then some additional memory transfers are required for communication between cores on different physical CPUs. Things are getting worse when one needs many state variables that are contained in unk(:,:,:,:,:) array (like dens, ener etc.). Paramesh developers decided to have variable index first, so all state quantities for a given cell occupy continuous region in memory. For solvers which use only some of the quantities this effects in poor CPU cache performance because lots of unnecessary data has to be loaded due to way how the cache works. For systems where all cores have own bus to memory the overhead in transfer may be hidden by well planned cache prefetch instructions. Systems with shared bus, like Quad Xeon, suffer here from more CPU stalls. New PARAMESH versions have utility to reorder unk and other arrays, but I have no idea how it can be utilized in FLASH. -- Cheers, Artur From latife at astro.rug.nl Tue Feb 12 04:16:39 2008 From: latife at astro.rug.nl (M.A. Latife) Date: Tue, 12 Feb 2008 11:16:39 +0100 Subject: [FLASH-USERS] isothermal behivour of gravitational collapse Message-ID: <47B17207.60600@astro.rug.nl> Hi all, Hope You will be fine and enjoying good health! once again i have a question. i am studying the gravitational collapse for 1-D spherical case and have introduced the effect of isothermal EOS.I am using the value of gamma 1.01 and have added cooling term to keep temperature constant (1*10^4K) .parameters i am using are given below. Every thing looks fine but still i am not getting 1/r^2 density profile as i should. density profile in my case is still 1/r^3. any body has experienced this problem. rho0 = 1.0E-25 T0= 1.0E+4 xmin = 0.0 xmax = 3.0E24 geometry = spherical kind regards M.A.Latif From flash-users at flash.uchicago.edu Wed Feb 13 11:18:53 2008 From: flash-users at flash.uchicago.edu (flash-users at flash.uchicago.edu) Date: Wed, 13 Feb 2008 11:18:53 -0600 (CST) Subject: [FLASH-USERS] [SPAM] RE: February %70 OFF Message-ID: <20080214111455.30840.qmail@host86-152-71-174.range86-152.btcentralplus.com> An HTML attachment was scrubbed... URL: http://flash.uchicago.edu/pipermail/flash-users/attachments/20080213/5810869a/attachment.html From mateuszr at umich.edu Thu Feb 21 22:28:23 2008 From: mateuszr at umich.edu (mateuszr at umich.edu) Date: Thu, 21 Feb 2008 23:28:23 -0500 Subject: [FLASH-USERS] Sedov scaling test In-Reply-To: References: <20080211135326.exjfnl9eb08w08g4@web.mail.umich.edu> Message-ID: <20080221232823.t1iw0meboggcw0s8@web.mail.umich.edu> Hi all, thanks very much for the help with the Sedov scaling test. Switching to mpich2 fixed the problem entirely and the scaling was practically linear. thanks again, Mateusz Quoting Anthony Chan : > > > On Mon, 11 Feb 2008, mateuszr at umich.edu wrote: > >> >> Hi all, >> >> I am testing a very small Dell cluster (8 cpus total). The machine >> is a dual quad core Xeon with 2 GB per core. The OS is Rocks 4.3, >> and the compilers used is the stock gcc 3.4.6 with mpich 1.2.7, >> mpif90 (Makefile.h enclosed below). FLASH version is 2.5 and I am >> using a Sedov test setup. >> >> The setup does not seem to scale with the number of processors at all. >> If anything, the execution time increases very slightly as I double >> the number of processors. I also noticed that all timesteps are >> printed out to the screen n times, where n is the number of >> processors. It seems to me that for some reason, the code just runs >> the same calculation n times instead of distributing the load >> evenly among all processors. > > It is posssible that you have n MPI_COMM_WORLD where each has only 1 > process (instead of 1 MPI_COMM_WORLD with n processes). You should > use MPICH2 which is more robust than MPICH1 especially in process management. > > A.Chan > >> >> It seems to me that this problem has nothing to do with the >> processor communication and, quite likely, is (embarrasingly?) >> trivial to solve. >> >> Has anybody seen this behavior before ? >> >> thanks, >> Mateusz >> >> >> P.S. I am enclosing Makefile.h below: >> >> # FLASH makefile definitions for ix86 Linux (Portland Group compiler) >> >> # note, in order to get pgprof to work properly, it was necessary to >> # download a new >> version from ftp://ftp.pgroup.com/ that plays nicely >> # with GNOME. >> >> #---------------------------------------------------------------------------- >> # Set the HDF/HDF5 library paths -- these need to be updated for your system >> #---------------------------------------------------------------------------- >> #HDF4_PATH = /usr/local/hdf4 >> #HDF5_PATH = /opt/HDF5-1.4.2-patch1-serial/ >> HDF5_PATH = /opt/packages/hdf5-1.6.6/ >> >> ZLIB_PATH = >> >> PAPI_PATH = >> PAPI_FLAGS = >> >> NCMPI_PATH = >> MPE_PATH = >> >> #---------------------------------------------------------------------------- >> # Compiler and linker commands >> # >> # Use the MPICH wrappers around the compilers -- these will automatically >> # load the proper libraries and include files. Version of MPICH prior >> # to 1.2.2 (?) do not recognize .F90 as a valid Fortran file extension. >> # You need to edit mpif90 and add .F90 to the test of filename extensions, >> # or upgrade your MPICH. >> #---------------------------------------------------------------------------- >> FCOMP = /opt/mpich/intel/bin/mpif90 >> CCOMP = /opt/mpich/intel/bin/mpicc >> CPPCOMP = /opt/mpich/intel/bin/mpicc >> LINK = /opt/mpich/intel/bin/mpif90 >> >> # pre-processor flag >> PP = -D >> >> #---------------------------------------------------------------------------- >> # Compilation flags >> # >> # Three sets of compilation/linking flags are defined: one for optimized >> # code, one for testing, and one for debugging. The default is to >> use the # _OPT >> version. Specifying -debug to setup will pick the _DEBUG version, >> # these should enable bounds checking. Specifying _TEST is used >> for # flash_test, and >> is set for quick code generation, and (sometimes) # profiling. The >> Makefile generated >> by setup will assign the generic token # (ex. FFLAGS) to the proper >> set of flags (ex. >> FFLAGS_OPT). >> #---------------------------------------------------------------------------- >> #FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/packages/hdf5-1.6.6/include >> -I/opt/mpich/intel/include/ -L/opt/mpich/intel/lib >> -L/opt/packages/hdf5-1.6.6/lib >> -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib >> -lhdf5 -lrfftw_mpi >> -lfftw_mpi -lrfftw >> FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/packages/hdf5-1.6.6/include >> -I/opt/mpich/intel/include/ -L/opt/mpich/intel/lib >> -L/opt/packages/hdf5-1.6.6/lib >> -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib >> -ldrfftw_mpi >> -ldfftw_mpi -ldrfftw -ldfftw >> # FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/usr/include/openmpi >> -I/usr/include/openmpi/64/ -L/usr/lib/openmpi >> #FFLAGS_OPT = -c -r8 -i4 -CB -traceback -I/opt/mpich/gnu/include >> -L/opt/mpich/gnu/lib >> FFLAGS_DEBUG = -g -c -Mbounds -r8 -i4 FFLAGS_TEST = -c -r8 -i4 -Mprof=lines >> >> F90FLAGS = >> >> #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include >> -I/usr/include/openmpi >> -I/usr/include/openmpi/64/ -L/usr/lib/openmpi >> #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include >> -I/opt/mpich/gnu/include >> -L/opt/mpich/gnu/lib >> #CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include >> -I/opt/intel/impi/3.1/include64/ >> -L/opt/intel/impi/3.1/lib64/ -L/opt/packages/hdf5-1.6.6/lib >> -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib >> -lhdf5 -lrfftw_mpi >> -lfftw_mpi -lrttfw >> CFLAGS_OPT = -O2 -c -I/opt/packages/hdf5-1.6.6/include >> -I/opt/intel/impi/3.1/include64/ >> -L/opt/intel/impi/3.1/lib64/ -L/opt/packages/hdf5-1.6.6/lib >> -I/opt/packages/fftw-2.1.5/include -L/opt/packages/fftw-2.1.5/lib >> -lhdf5 -ldrfftw_mpi >> -ldfftw_mpi -ldrfftw -ldfftw >> >> CFLAGS_DEBUG = -g -c >> CFLAGS_TEST = -c >> >> # if we are using HDF5, we need to specify the path to the include files >> CFLAGS_HDF5 = -I $(HDF5_PATH)/include >> >> CFLAGS_NCMPI = >> >> #---------------------------------------------------------------------------- >> # Linker flags >> # >> # There is a seperate version of the linker flags for each of the >> _OPT, # _DEBUG, and >> _TEST cases. >> #---------------------------------------------------------------------------- >> >> LFLAGS_OPT = -o >> LFLAGS_DEBUG = -g -o >> LFLAGS_TEST = -Mprof=lines -o >> >> >> #---------------------------------------------------------------------------- >> # Library specific linking >> # >> # If a FLASH module has a 'LIBRARY xxx' line in its Config file, we need to >> # create a macro in this Makefile.h for LIB_xxx, which will be added to the >> # link line when FLASH is built. This allows us to switch between >> different >> # (incompatible) libraries. We also create a _OPT, _DEBUG, and _TEST >> # library macro to add any performance-minded libraries (like fast math), >> # depending on how FLASH was setup. >> #---------------------------------------------------------------------------- >> >> LIB_OPT = LIB_DEBUG = >> LIB_TEST = >> >> >> #LIB_MATH = -L/u/mateusz/fftw-2.1.5/lib -ldrfftw_mpi -ldfftw_mpi >> -ldrfftw -ldfftw >> #LIB_MATH = -L/opt/packages/fftw-2.1.5/lib -ldrfftw_mpi -ldfftw_mpi >> -ldrfftw -ldfftw >> LIB_MATH = -L/opt/packages/fftw-2.1.5/lib >> -L/opt/packages/hdf5-1.6.6/lib -lrfftw_mpi >> -lfftw_mpi -lrfftw -lfftw -lhdf5 #LIB_HDF4 = -L $(HDF4_PATH)/lib >> -lmfhdf -ldf -ljpeg -lz >> LIB_HDF5 = -L $(HDF5_PATH)/lib $(LIB_MATH) -lmpi -lz -lsz >> #LIB_HDF5 = -lhdf5 $(LIB_MATH) -lmpi >> >> LIB_PAPI = >> >> LIB_NCMPI = >> LIB_MPE = >> >> #---------------------------------------------------------------------------- >> # Additional machine-dependent object files >> # >> # Add any machine specific files here -- they will be compiled and linked >> # when FLASH is built. >> #---------------------------------------------------------------------------- >> >> MACHOBJ = >> >> >> #---------------------------------------------------------------------------- >> # Additional commands >> #---------------------------------------------------------------------------- >> >> MV = mv -f >> AR = ar -r >> RM = rm -f >> CD = cd >> RL = ranlib >> ECHO = echo >> >> >> > > > From dubey at flash.uchicago.edu Fri Feb 29 15:28:05 2008 From: dubey at flash.uchicago.edu (Anshu Dubey) Date: Fri, 29 Feb 2008 15:28:05 -0600 (CST) Subject: [FLASH-USERS] FLASH 3.0 release Message-ID: <59420.128.135.152.186.1204320485.squirrel@flash.uchicago.edu> The Flash Center is pleased to announce the release of the next major version of the FLASH code, version 3.0. FLASH 3.0 offers many advantages over the FLASH 2.5 release. The more significant advantages of FLASH 3.0 are : * Clean public interfaces for each code unit. * Increased modularity through better encapsulation of code units, making it considerably easier to incorporate third party software into FLASH. * Support for a uniform grid. * Support for face variables. * Support for defining blocksizes at runtime. This feature can enable FLASH to use meshes that do not require all blocks be equal in size. * Ability to switch between uniform and adaptive Grid for the same application. * Support for directionally un-split time integration. * Unit test framework and many unit tests. * Significantly enhanced setup script, see Chapter 5 in the user's guide for details. * New Staggered Mesh unsplit MHD solver. * New, more efficient multigrid algorithm. * More scalable algorithms for Particles management. * Significant optimization of the multipole algorithm for many problems. * More robust and efficient IO implementation. * Extensive web published API documentation for all units in the code. Capabilities added since the beta release include: * Full support for active particles, including new, more scalable algorithms for mapping the particles to the mesh. * A new multigrid solver for self gravity (courtesy of Paul Ricker). * The Cosmology unit * The Relativistic Hydrodynamics Unit * A parallel FFT solver that can be used with Uniform grid * Many simulation setups and unit tests. The FLASH3.0 release is available for download with a license agreement at: http://flash.uchicago.edu/website/download/ A stripped down version of FLASH3 that may be downloaded without a license is also available at the same location. This version is essentially the FLASH framework without any implementations. Flash Center is also providing support for "add-ons" to the code. Please see the section on "What's new in this release" in the first chapter of the user's guide for details. Additionally, the FLASH testing software FlashTest, which became available with the alpha release, continues to be available for download at: http://flash.uchicago.edu/website/codesupport/ Many, but not all parts of FLASH3 are backwards-compatible with FLASH2. The Flash code group has written extensive documentation detailing how to make the transition from FLASH2 to FLASH3 as smooth as possible. The user should look to: http://flash.uchicago.edu/website/codesupport/ The website also contains other documentation including a user's guide and a developer's section. A new feature in FLASH3 documentation is the online description of the public interface routines to various code units. FLASH should be portable to most UNIX-like operating systems with a python interpreter, Fortran 90 compiler, C compiler and MPI library. Please see the RELEASE-NOTES included in the FLASH source home directory for a list of compilers, platforms and library versions that are known to be compatible with FLASH. FLASH output can be visualized using VisIt, a free interactive visualization tool developed by Lawrence Livermore National Laboratories under the ASCI initiative. VisIt is available from http://flash.uchicago.edu/website/codesupport/visit, and https://wci.llnl.gov/codes/visit To use the current released version of Visit with the FLASH file format version 9 (the 3.0 release version) users will need a patch which is also availble from http://flash.uchicago.edu/website/codesupport/visit Development of the FLASH Code was funded by the DOE-supported ASC/Alliance Center for Astrophysical Thermonuclear Flashes. We acknowledge support received from Lawrence Livermore National Laboratory and the University of Chicago. All publications resulting from the use of the FLASH Code must acknowledge the ASC/Alliance Center for Astrophysical Thermonuclear Flashes. Addition of the following text to the paper acknowledgments will be sufficient. "The software used in this work was in part developed by the DOE-supported ASC/Alliance Center for Astrophysical Thermonuclear Flashes at the University of Chicago."