From flash-users at flash.uchicago.edu  Tue Mar  3 17:46:02 2009
From: flash-users at flash.uchicago.edu (flash-users at flash.uchicago.edu)
Date: Tue,  3 Mar 2009 17:46:02 -0600 (CST)
Subject: [FLASH-USERS] USA Menshealth ID326911251 Maddona\'s private
	party
Message-ID: <20090303234603.278023401C@flash.uchicago.edu>

An HTML attachment was scrubbed...
URL: http://flash.uchicago.edu/pipermail/flash-users/attachments/20090303/7c4a9be3/attachment.html 

From mh475 at cam.ac.uk  Wed Mar 11 06:43:13 2009
From: mh475 at cam.ac.uk (Martin Huarte-Espinosa)
Date: Wed, 11 Mar 2009 11:43:13 +0000
Subject: [FLASH-USERS] 3DMHD +usm, jumps of divb at refinement jumps
In-Reply-To: <5b942a610902200826r5f2f5636x9b423462886a178d@mail.gmail.com>
References: <3182e9f70902200156p53b8a99fwf92689766ee12a4f@mail.gmail.com>
	<499ECF32.60608@flash.uchicago.edu>
	<3182e9f70902200759i71551621k3ef32571b72a6d0f@mail.gmail.com>
	<5b942a610902200826r5f2f5636x9b423462886a178d@mail.gmail.com>
Message-ID: <3182e9f70903110443j3a22bd9aocb1410899607f998@mail.gmail.com>

Good day Flash users.

I've fixed the mhd problem. The solution:

At Simulation_initBlock I read the magnetic vector potential and both, the
vector and the code's grid, should have resolution. This must be flash's
coarse gird for nblockx = 1, nblocky = 1, nblockz = 1 and
lrefine_max=lrefine_min=4, say.  Then, I run the code for a few timesteps,
stop it and restart from the produce chk file, allowing lrefine_max>4 ONLY.
This maintains a dvib within ~ +- 1.5e-11.

Cheers.


Best,

Martin HE.


On Fri, Feb 20, 2009 at 4:26 PM, Anshu Dubey <dubey at flash.uchicago.edu>wrote:

> Non fixed blocksize UG mode by default uses parallel IO. The output is
> written as a
> single large block, and that is the only way to restart on a different
> number of
> processors, and that mode has to be necessarily parallel. You can
> either use parallel
> hdf5, or switch to using fixed blocksize UG, which does have serial IO
> support.
> In principle there is nothing that prevents serial IO from being used
> with nonfixedblocksize
> mode, but some Config files may need tweaking to enable that. We will
> look into it
> and get back to you.
>
> You have to keep in mind that if you use serial IO, then for all
> practical purposes
> your mode reduces to fixed blocksize mode if you wish to restart. You
> can restart
> on exactly the same number of processors, and in exactly the same
> configuration
> of those processors, as the original from_scratch run.
>
> Anshu
>
> On Fri, Feb 20, 2009 at 9:59 AM, Martin Huarte-Espinosa <mh475 at cam.ac.uk>
> wrote:
> > Hi Chris and thanks for your answer. I've tried the following,
> > unsuccessfully:
> >
> > ./setup /Sysims/mhd_ug_nofbs -3d +usm -auto -opt +nofbs +serialIO
> > -objdir=mhd_ug_nofbs -site=apgrid && cd mhd_ug_nofbs/ && make
> >
> > Best,
> >
> > Martin HE.
> >
> >
> > On Fri, Feb 20, 2009 at 3:41 PM, Chris Daley <cdaley at flash.uchicago.edu>
> > wrote:
> >>
> >> This is only a response to your first problem.  I think FLASH is
> >> failing to link because your HDF5 build does not have parallel IO
> >> support. If you would like to use parallel IO, re-build HDF5 with the
> >> additional configure options "--enable-parallel CC=mpicc".  If not,
> >> then simply add the shortcut +serialIO during FLASH setup for serial IO.
> >>
> >> Regards,
> >> Chris
> >>
> >> Martin Huarte-Espinosa wrote:
> >>>
> >>> Good day Flash community:
> >>>
> >>> Two questions. I'll be really thankful if you had any hints about them.
> >>>
> >>> 1) I'm trying to complie flash3.1 for a uniform grid with no fixed
> block
> >>> size:
> >>>
> >>> ./setup mhd_ug_nofbs -3d +usm -auto -opt +nofbs -site=apgrid
> >>>
> >>> and I get:
> >>>
> >>> /-L/mraosw/data1/mh475/krause/lib/hdf/5-1.6.5-amd-icc/lib -lhdf5 -lz
> >>> -lhdf5_fortran -L/mraosw/data1/mh475/krause/lib/mpich-1.2.6/lib -lmpich
> >>>
>  /home/krause/dataw/lib/mpich-1.2.4/lib/libmpich.a(p4_secure.o)(.text+0x91):
> >>> In function `start_slave':
> >>> : warning: Using 'getpwuid' in statically linked applications requires
> at
> >>> runtime the shared libraries from the glibc version used for linking
> >>>
> >>>
> /mraosw/data1/mh475/krause/lib/hdf/5-1.6.5-amd-icc/lib/libhdf5.a(H5FDstream.o)(.text+0x723):
> >>> In function `H5FD_stream_open_socket':
> >>> : warning: Using 'gethostbyname' in statically linked applications
> >>> requires at runtime the shared libraries from the glibc version used
> for
> >>> linking
> >>> io_h5file_interface.o(.text+0x6d): In function `io_h5init_file_':
> >>> : undefined reference to `H5Pset_fapl_mpio'
> >>> io_h5file_interface.o(.text+0x12f): In function
> >>> `io_h5open_file_for_read_':
> >>> : undefined reference to `H5Pset_fapl_mpio'
> >>> make: *** [flash3] Error 1/
> >>>
> >>> I've tried, unsuccessfully, to edit some parts of the cluster's
> >>> Makefile.h.
> >>>
> >>>
> >>> 2) I'm implementing a gas with a random magnetic distribution. I
> generate
> >>> the magnetic fields outside flash, making sure their divb~1.e-14. I
> then
> >>> read these fields into Flash3.1 and they work well for a uniform grid
> or
> >>> with the AMR with lrefine_max=lrefine_min. However, if I give
> lrefine_max
> >>> different from lrefine_min, divb appears to be fine at time=0, but
> after the
> >>> first timestep it gets jumps up to ~ 0.1 at cells adjacent to
> refinement
> >>> jumps only. I'm attaching important bits of info below.
> >>>
> >>> Thanks a lot.
> >>>
> >>> ./setup MySims/mhd_01 -3d +usm -auto  -opt -maxblocks=400 -site=darwin
> >>>
> >>> ------------------------
> >>> /subroutine Simulation_initBlock(blockID, myPE, F) ! following the
> setup
> >>> from the Fields loop test problem/
> >>>
> >>> /  use Simulation_data, ONLY :   sim_gCell,  sim_gamma,   &/
> >>> /                                sim_smallX, sim_smallP,  &/
> >>> /                                sim_beta,   sim_B_cluster,     &/
> >>> /        sim_killdivb, sim_rc,   &/
> >>> /sim_xMin, sim_xMax, sim_yMin, &/
> >>> /sim_yMax, sim_zMin, sim_zMax, &/
> >>> /sim_n_cube/
> >>>
> >>> /!  use Grid_data, ONLY :        gr_nBlockX, gr_nBlockY, gr_nBlockZ/
> >>>
> >>> /!  use tree, ONLY : lrefine_max, lrefine_min/
> >>>
> >>> /  use Grid_interface, ONLY : Grid_getBlkIndexLimits, &/
> >>> /                             Grid_getCellCoords,     &/
> >>> /                             Grid_getBlkPtr,         &/
> >>> /                             Grid_releaseBlkPtr/
> >>>
> >>> /  use Driver_interface, ONLY : Driver_abortFlash/
> >>> /  /
> >>> /  implicit none/
> >>>
> >>> /#include "constants.h"/
> >>> /#include "Flash.h"/
> >>>
> >>> /  !!$ Arguments -----------------------/
> >>> /  integer, intent(in) :: blockID, myPE/
> >>> /  !!$ ---------------------------------/
> >>>
> >>> /  integer :: i, j, k, n, istat, sizeX, sizeY, sizeZ/
> >>> /  integer, dimension(2,MDIM) :: blkLimits, blkLimitsGC/
> >>> /  real :: enerZone, ekinZone, eintZone/
> >>> /  real :: rot, radius, dx, dy, dz, r0, taper, vel_factor/
> >>> /  real, allocatable,dimension(:) :: xCoord,xCoordL,xCoordR,&/
> >>> /                                    yCoord,yCoordL,yCoordR,&/
> >>> /                                    zCoord,zCoordL,zCoordR/
> >>> /  real, dimension(MDIM) :: del/
> >>> /  real, pointer, dimension(:,:,:,:) :: solnData, facexData, faceyData,
> >>> facezData/
> >>> /  real :: xx,yy,zz/
> >>> /#ifdef FIXEDBLOCKSIZE/
> >>> /  real, dimension(GRID_IHI_GC+1,GRID_JHI_GC+1,GRID_KHI_GC+1) ::
> >>> Az,Ax,Ay/
> >>> / //#else /
> >>> /  real, allocatable, dimension(:,:,:) :: Az,Ax,Ay/
> >>> / //#endif /
> >>>
> >>> /logical, save :: once=.true., once2=.true., VecPotA=.true./
> >>> /        integer :: ii, jj, kk/
> >>> /        real :: B0, x_slope, y_slope, z_slope, &/
> >>> /                idl_x_ini, idl_x_end, &/
> >>> /                idl_y_ini, idl_y_end, &/
> >>> /                idl_z_ini, idl_z_end, &/
> >>> /largestCell, minimumX, maximumX, &/
> >>> /     minimumY, maximumY ,&/
> >>> /     minimumZ, maximumZ, &/
> >>> /                stdev_b, max_b, mean_pres, mean_b, norm_b
> >>>
> >>> //        real, dimension(3,sim_n_cube,sim_n_cube,sim_n_cube),
> intent(in)
> >>>  :: F
> >>>        !contains the previously-generated magnetic fields
> >>> //!!/
> >>>
> >>>
> >>> /  ! dump some output to stdout listing the paramters/
> >>> /  if (myPE == MASTER_PE) then/
> >>> /1    format (1X, 1P, 4(A7, E13.7, :, 1X))/
> >>> /2    format (1X, 1P, 2(A7, E13.7, 1X), A7, I13)/
> >>> /  endif/
> >>>
> >>> /  call Grid_getBlkIndexLimits(blockId,blkLimits,blkLimitsGC)/
> >>>
> >>> /  sizeX = blkLimitsGC(HIGH,IAXIS)-blkLimitsGC(LOW,IAXIS)+1/
> >>> /  sizeY = blkLimitsGC(HIGH,JAXIS)-blkLimitsGC(LOW,JAXIS)+1/
> >>> /  sizeZ = blkLimitsGC(HIGH,KAXIS)-blkLimitsGC(LOW,KAXIS)+1/
> >>>
> >>> /  allocate(xCoord(sizeX), stat=istat)/
> >>> /  allocate(xCoordL(sizeX),stat=istat)/
> >>> /  allocate(xCoordR(sizeX),stat=istat)/
> >>>
> >>> /  allocate(yCoord(sizeY), stat=istat)/
> >>> /  allocate(yCoordL(sizeY),stat=istat)/
> >>> /  allocate(yCoordR(sizeY),stat=istat)/
> >>>
> >>> /  allocate(zCoord(sizeZ), stat=istat)/
> >>> /  allocate(zCoordL(sizeZ),stat=istat)/
> >>> /  allocate(zCoordR(sizeZ),stat=istat)/
> >>>
> >>> /  xCoord  = 0.0/
> >>> /  xCoordL = 0.0/
> >>> /  xCoordR = 0.0/
> >>>
> >>> /  yCoord  = 0.0/
> >>> /  yCoordL = 0.0/
> >>> /  yCoordR = 0.0/
> >>>
> >>> /  zCoord  = 0.0/
> >>> /  zCoordL = 0.0/
> >>> /  zCoordR = 0.0/
> >>>
> >>> /#ifndef FIXEDBLOCKSIZE/
> >>> /  if (NDIM == 2) then/
> >>> /     allocate(Ax(sizeX+1,sizeY+1,1),stat=istat)/
> >>> /     allocate(Ay(sizeX+1,sizeY+1,1),stat=istat)/
> >>> /     allocate(Az(sizeX+1,sizeY+1,1),stat=istat)/
> >>> /     //  elseif (NDIM == 3) then/
> >>> /     allocate(Ax(sizeX+1,sizeY+1,sizeZ+1),stat=istat)/
> >>> /     allocate(Ay(sizeX+1,sizeY+1,sizeZ+1),stat=istat)/
> >>> /     allocate(Az(sizeX+1,sizeY+1,sizeZ+1),stat=istat)/
> >>> /     //  endif/
> >>> /#endif/
> >>>
> >>>
> >>> /  if (NDIM == 3) then/
> >>> /   call Grid_getCellCoords(KAXIS,blockId,CENTER,    sim_gCell,zCoord,
> >>> sizeZ)/
> >>> /   call Grid_getCellCoords(KAXIS,blockId,LEFT_EDGE,
> >>> sim_gCell,zCoordL,sizeZ)/
> >>> /   call
> >>> Grid_getCellCoords(KAXIS,blockId,RIGHT_EDGE,sim_gCell,zCoordR,sizeZ)/
> >>> /  endif/
> >>> /  if (NDIM >= 2) then/
> >>> /     call Grid_getCellCoords(JAXIS,blockId,CENTER,
>  sim_gCell,yCoord,
> >>> sizeY)/
> >>> /     call Grid_getCellCoords(JAXIS,blockId,LEFT_EDGE,
> >>> sim_gCell,yCoordL,sizeY)/
> >>> /     call
> >>> Grid_getCellCoords(JAXIS,blockId,RIGHT_EDGE,sim_gCell,yCoordR,sizeY)/
> >>> /  endif/
> >>>
> >>> /  call Grid_getCellCoords(IAXIS,blockId,CENTER,    sim_gCell,xCoord,
> >>> sizeX)/
> >>> /  call Grid_getCellCoords(IAXIS,blockId,LEFT_EDGE,
> >>> sim_gCell,xCoordL,sizeX)/
> >>> /  call
> >>> Grid_getCellCoords(IAXIS,blockId,RIGHT_EDGE,sim_gCell,xCoordR,sizeX)/
> >>>
> >>> /  call Grid_getDeltas(blockID,del)/
> >>> /  dx = del(1)/
> >>> /  dy = del(2)/
> >>> /  dz = del(3)/
> >>>
> >>> /
> >>>
>  !------------------------------------------------------------------------------/
> >>> /  ! Construct Az at each cell corner/
> >>> /  ! Bx = dAz/dy - dAy/dz/
> >>> /  ! By = dAx/dz - dAz/dx/
> >>> /  ! Bz = dAy/dx - dAx/dy/
> >>> /  Az = 0./
> >>> /  Ax = 0./
> >>> /  Ay = 0.
> >>>
> >>>  x_ini = 1.
> >>>  y_ini = 1.    z_ini = 1.
> >>>  x_end = real(sim_n_cube)
> >>>  y_end = real(sim_n_cube)
> >>>  z_end = real(sim_n_cube)
> >>>                   minimumX = sim_xMin - 4.d0*dx
> >>>           minimumY = sim_yMin - 4.d0*dy
> >>>           minimumZ = sim_zMin - 4.d0*dz
> >>>             maximumX = sim_xMax + 4.d0*dx
> >>>             maximumY = sim_yMax + 4.d0*dy
> >>>             maximumZ = sim_zMax + 4.d0*dz /
> >>> /           /
> >>> /   x_slope = (x_end-x_ini)/(maximumX-minimumX)/
> >>> /   y_slope = (y_end-y_ini)/(maximumY-minimumY) /
> >>> /   z_slope = (z_end-z_ini)/(maximumZ-minimumZ) /
> >>> /    /
> >>> /     do k = blkLimitsGC(LOW,KAXIS),blkLimitsGC(HIGH,KAXIS)+1/
> >>> /        do j = blkLimitsGC(LOW,JAXIS),blkLimitsGC(HIGH,JAXIS)+1/
> >>> /           do i = blkLimitsGC(LOW,IAXIS),blkLimitsGC(HIGH,IAXIS)+1/
> >>>
> >>>
> >>> /              ! x Coord at cell corner/
> >>> /              if (i <=blkLimitsGC(HIGH,IAXIS)) then/
> >>> /                 xx = xCoordL(i)/
> >>> /              else/
> >>> /                 xx = xCoordR(i-1)/
> >>> /              endif/
> >>>
> >>> /              ! y Coord at cell corner/
> >>> /              if (j <=blkLimitsGC(HIGH,JAXIS)) then/
> >>> /                 yy = yCoordL(j)/
> >>> /              else/
> >>> /                 yy = yCoordR(j-1)/
> >>> /              endif/
> >>>
> >>> /              ! z Coord at cell corner/
> >>> /              if (k <=blkLimitsGC(HIGH,KAXIS)) then/
> >>> /                 zz = zCoordL(k)/
> >>> /              else/
> >>> /                 zz = zCoordR(k-1)/
> >>> /              endif/
> >>>
> >>>
> >>> /              ii = nint(  x_slope*(xx-minimumX)+idl_x_ini  ) /
> >>> /              jj = nint(  y_slope*(yy-minimumY)+idl_y_ini  )/
> >>> /              kk = nint(  z_slope*(zz-minimumZ)+idl_z_ini  )/
> >>>
> >>> /!--------------------------------------------------------------/
> >>> /if ( (ii.lt.idl_x_ini).or.(ii.gt.idl_x_end).or.&/
> >>> /     (jj.lt.idl_y_ini).or.(jj.gt.idl_y_end).or.& /
> >>> /     (kk.lt.idl_z_ini).or.(kk.gt.idl_z_end)  ) &/
> >>> /call  Driver_abortFlash("*** init_block, IDL-Flash coords. error
> ***")/
> >>>
> >>> /!!--------------------------------------------------------------/
> >>>
> >>> !Read the outise-fields into flash
> >>> /                 Ax(i,j,k) = F(1,ii,jj,kk)/
> >>> /                 Ay(i,j,k) = F(2,ii,jj,kk)/
> >>> /                 Az(i,j,k) = F(3,ii,jj,kk)/
> >>> /      /
> >>> /           enddo/
> >>> /        enddo/
> >>> /     enddo/
> >>>
> >>>
> >>> /!Initial conditions:/
> >>>
> >>> /  call Grid_getBlkPtr(blockID,solnData,CENTER)/
> >>>
> >>> /#if NFACE_VARS > 0/
> >>> /  if (sim_killdivb) then/
> >>> /     call Grid_getBlkPtr(blockID,facexData,FACEX)/
> >>> /     call Grid_getBlkPtr(blockID,faceyData,FACEY)/
> >>> /     if (NDIM == 3) call Grid_getBlkPtr(blockID,facezData,FACEZ)/
> >>> /  endif/
> >>> /#endif/
> >>>
> >>>
> >>> /  ! Loop over cells within the block./
> >>> /  do k = blkLimitsGC(LOW,KAXIS),blkLimitsGC(HIGH,KAXIS)/
> >>> /     do j = blkLimitsGC(LOW,JAXIS),blkLimitsGC(HIGH,JAXIS)/
> >>> /        do i = blkLimitsGC(LOW,IAXIS),blkLimitsGC(HIGH,IAXIS)/
> >>>
> >>> /
> solnData(SPECIES_BEGIN,i,j,k)=1.0e0-(NSPECIES-1)*sim_smallX/
> >>> /           do n=SPECIES_BEGIN,SPECIES_END/
> >>> /              solnData(n,i,j,k)=sim_smallX/
> >>> /           enddo/
> >>> / /
> >>>
> >>> /           solnData(DENS_VAR,i,j,k)= 1./
> >>> /           solnData(PRES_VAR,i,j,k)=
> solnData(DENS_VAR,i,j,k)/sim_gamma
> >>> /
> >>>
> >>>
> >>> /           solnData(VELX_VAR,i,j,k)= 0.d0/
> >>> /           solnData(VELY_VAR,i,j,k)= 0.d0/
> >>> /           solnData(VELZ_VAR,i,j,k)= 0.d0/
> >>>
> >>> /           /
> >>> / /
> >>>
> >>> /           ! Compute the gas energy and set the gamma-values needed
> for
> >>> the EOS/
> >>> /           ekinZone = 0.5 *
> >>> dot_product(solnData(VELX_VAR:VELZ_VAR,i,j,k),&/
> >>> /
> >>>  solnData(VELX_VAR:VELZ_VAR,i,j,k))/
> >>>
> >>> /           ! specific internal energy/
> >>> /           eintZone =
> >>> solnData(PRES_VAR,i,j,k)/(sim_gamma-1.)/solnData(DENS_VAR,i,j,k)/
> >>>
> >>> /           ! total specific gas energy/
> >>> /           enerZone = eintZone + ekinZone/
> >>>
> >>> /           ! Take a limit value/
> >>> /           enerZone = max(enerZone, sim_smallP)/
> >>>
> >>> /           solnData(ENER_VAR,i,j,k)=enerZone/
> >>> /           solnData(EINT_VAR,i,j,k)=eintZone/
> >>> /           solnData(GAMC_VAR,i,j,k)=sim_gamma/
> >>> /           solnData(GAME_VAR,i,j,k)=sim_gamma/
> >>>
> >>>
> >>>
> >>> /        enddo/
> >>> /     enddo/
> >>> /  enddo/
> >>>
> >>>
> >>> /!! CURL:/
> >>> /  do k = blkLimitsGC(LOW,KAXIS),blkLimitsGC(HIGH,KAXIS)+1/
> >>> /     do j = blkLimitsGC(LOW,JAXIS),blkLimitsGC(HIGH,JAXIS)+1/
> >>> /        do i = blkLimitsGC(LOW,IAXIS),blkLimitsGC(HIGH,IAXIS)+1/
> >>>
> >>>
> >>> /              if ( (j <=blkLimitsGC(HIGH,JAXIS)).and.&/
> >>> /           (k <=blkLimitsGC(HIGH,KAXIS))  ) then/
> >>> /                 facexData(MAG_FACE_VAR,i,j,k)= -(Ay(i  ,j
> >>>  ,k+1)-Ay(i,j,k))/dz &/
> >>> /+(Az(i  ,j+1,k  )-Az(i,j,k))/dy/
> >>> /      end if/
> >>>
> >>> /              if ( (i <=blkLimitsGC(HIGH,IAXIS)).and.&/
> >>> /           (k <=blkLimitsGC(HIGH,KAXIS))  ) then/
> >>> /                 faceyData(MAG_FACE_VAR,i,j,k)=  (Ax(i  ,j
> >>>  ,k+1)-Ax(i,j,k))/dz &/
> >>> /-(Az(i+1,j  ,k  )-Az(i,j,k))/dx/
> >>> /      end if/
> >>>
> >>> /              if ( (j <=blkLimitsGC(HIGH,JAXIS)).and.&/
> >>> /           (i <=blkLimitsGC(HIGH,IAXIS))  ) then/
> >>> /                 facezData(MAG_FACE_VAR,i,j,k)= -(Ax(i  ,j+1,k
> >>>  )-Ax(i,j,k))/dy &/
> >>> /+(Ay(i+1,j  ,k  )-Ay(i,j,k))/dx/
> >>> /      end if/
> >>> /!N end/
> >>>
> >>> /      enddo/
> >>> /     enddo/
> >>> /  enddo/
> >>>
> >>> / /!!B, DIVB AND MAGP:
> >>> /  do k=blkLimitsGC(LOW,KAXIS),blkLimitsGC(HIGH,KAXIS)/
> >>> /     do j = blkLimitsGC(LOW,JAXIS),blkLimitsGC(HIGH,JAXIS)/
> >>> /        do i = blkLimitsGC(LOW,IAXIS),blkLimitsGC(HIGH,IAXIS)
> >>>
> >>> /
> >>> /           solnData(MAGX_VAR,i,j,k) =
> >>> 0.5*(facexData(MAG_FACE_VAR,i,j,k)+facexData(MAG_FACE_VAR,i+1,j,k))/
> >>> /           solnData(MAGY_VAR,i,j,k) =
> >>> 0.5*(faceyData(MAG_FACE_VAR,i,j,k)+faceyData(MAG_FACE_VAR,i,j+1,k))/
> >>> /           if (NDIM == 3) then/
> >>> /              solnData(MAGZ_VAR,i,j,k) =
> >>> 0.5*(facezData(MAG_FACE_VAR,i,j,k)+facezData(MAG_FACE_VAR,i,j,k+1))/
> >>> /           endif/
> >>>
> >>>
> >>> /          /
> >>> /         solnData(DIVB_VAR,i,j,k)= &/
> >>> /                     (facexData(MAG_FACE_VAR,i+1,j,  k  ) -
> >>> facexData(MAG_FACE_VAR,i,j,k))/dx &/
> >>> /                   + (faceyData(MAG_FACE_VAR,i,  j+1,k  ) -
> >>> faceyData(MAG_FACE_VAR,i,j,k))/dy/
> >>>
> >>> /           solnData(DIVB_VAR,i,j,k)= solnData(DIVB_VAR,i,j,k) &/
> >>> /                   + (facezData(MAG_FACE_VAR,i,  j,  k+1) -
> >>> facezData(MAG_FACE_VAR,i,j,k))/dz/
> >>>
> >>>
> >>> /           solnData(MAGP_VAR,i,j,k) =
> >>> .5*dot_product(solnData(MAGX_VAR:MAGZ_VAR,i,j,k),&/
> >>> /
> >>> solnData(MAGX_VAR:MAGZ_VAR,i,j,k))/
> >>>
> >>>
> >>> /        enddo/
> >>> /     enddo/
> >>> /  enddo/
> >>>
> >>>
> >>>
> >>> /  ! Release pointer/
> >>> /  call Grid_releaseBlkPtr(blockID,solnData,CENTER)/
> >>>
> >>> /#if NFACE_VARS > 0/
> >>> /  if (sim_killdivb) then/
> >>> /     call Grid_releaseBlkPtr(blockID,facexData,FACEX)/
> >>> /     call Grid_releaseBlkPtr(blockID,faceyData,FACEY)/
> >>> /     if (NDIM == 3) call Grid_releaseBlkPtr(blockID,facezData,FACEZ)/
> >>> /  endif/
> >>> /#endif/
> >>>
> >>> /deallocate(xCoord)/
> >>> /deallocate(xCoordL)/
> >>> /deallocate(xCoordR)/
> >>> /!/
> >>> /deallocate(yCoord)/
> >>> /deallocate(yCoordL)/
> >>> /deallocate(yCoordR)/
> >>>
> >>> /deallocate(zCoord)/
> >>> /deallocate(zCoordL)/
> >>> /deallocate(zCoordR)/
> >>>
> >>> /#ifndef FIXEDBLOCKSIZE/
> >>> /  deallocate(Az)/
> >>> /  deallocate(Ax)/
> >>> /  deallocate(Ay)/
> >>> /#endif/
> >>>
> >>> /end subroutine Simulation_initBlock/
> >>>
> >>> ---------------------------------
> >>> flash.par:
> >>>
> >>>
> >>> sim_n_cube = 73 #= 64+8guard+1curl
> >>> nblockx = 1
> >>> nblocky = 1
> >>> nblockz = 1
> >>> lrefine_max     = 5
> >>> lrefine_min     =  3
> >>> refine_var_1    = "velx"
> >>> nrefs           = 2
> >>>
> >>> run_comment     = "14Feb09-1"
> >>> log_file        = "14Feb09-1.log"
> >>> basenm          = "14Feb09-1-"
> >>> restart             = .false.
> >>> nend                 = 10
> >>> checkPointFileNumber = 1
> >>> tmax            = 4.0
> >>> checkpointFileIntervalTime = 1.0
> >>> plotFileIntervalTime    = 8.e-4 #2.5e-2
> >>> dtmax                   = 0.02  #0.01
> >>> plotFileNumber  = 0
> >>>
> >>> sim_beta                = 100.0     #Magnetic beta
> >>>
> >>> gamma           = 1.666666666667
> >>>
> >>> geometry       = "cartesian"
> >>>
> >>> xmin            = -0.5
> >>> xmax            =  0.5
> >>> ymin            = -0.5
> >>> ymax            =  0.5
> >>> zmin            = -0.5
> >>> zmax            =  0.5
> >>> xl_boundary_type = "outflow"
> >>> xr_boundary_type = "outflow"
> >>> yl_boundary_type = "outflow"
> >>> yr_boundary_type = "outflow"
> >>> zl_boundary_type = "outflow"
> >>> zr_boundary_type = "outflow"
> >>>
> >>> #       Simulation (grid, time, I/O) parameters
> >>>
> >>> cfl             = 0.3
> >>> dtini           = 1.e-12
> >>> plot_var_1      = "dens"
> >>> plot_var_2      = "velx"
> >>> plot_var_3      = "vely"
> >>> plot_var_4      = "velz"
> >>> plot_var_5      = "pres"
> >>> plot_var_6      = "magx"
> >>> plot_var_7      = "magy"
> >>> plot_var_8      = "magz"
> >>> plot_var_9      = "divb"
> >>> plot_var_10     = "magp"
> >>>
> >>> convertToConsvdInMeshInterp = .true.
> >>>
> >>> eintSwitch      = 1.e-6
> >>>
> >>> #MHD:
> >>> ForceHydroLimit = .false. #(set .true. when using hydro limit B=0)
> >>> sim_B_cluster   = 1.0
> >>> UnitSystem      = "none"
> >>> resistive_mhd   = .false.
> >>> killdivb        = .true.
> >>>
> >>> flux_correct    = .true.
> >>>
> >>> order           = 2 #(first / second order scheme)
> >>> slopeLimiter    = "vanLeer" #(minmod, mc, vanLeer, hybrid, limited)
> >>> LimitedSlopeBeta= 1.   # only needed for the limited slope by Toro
> >>> charLimiting    = .true. #(.false. will give primitive limitings)
> >>> E_modification  = .true. #(.false. will use simple arithmetic avg)
> >>> energyFix       = .true. #(.false. will not fix total energy due to
> >>> div-free magnetic pressure)
> >>> facevar2ndOrder        = .true. #(.false. will give less accurate
> >>> solution but fast performance)
> >>>
> >>> RiemannSolver   = "hll"
> >>>  #       CTU integrator
> >>> CTU             = .false. #(6-solve CTU scheme, not recommended)
> >>>
> >>> #       Prolongation method of Facevars
> >>> prolMethod      = "injection_prol" #(injecton_prol, balsara_prol)
> >>>
> >>> #ICM initial conditions with a central initial jet /10:
> >>> smlrho          = 1.0e-4 # dens_j/10.
> >>> smallp          = 1.e-3 # ~ min(pres)_ICM/10
> >>> smalle          = 0.00900001 #(1/gamma)/((gamma-1)*1)/10: initial IMC
> >>> Therm Energy/10
> >>>                #15.0
> >>> #smallt          = 1.000
> >>>
> >>> #Total energy minimum limit
> >>> smallu          = 0.000900001 #[(1/gamma)/((gamma-1)*1) + 0.]/10
> initial
> >>> IMC Total Energy/10
> >>>                #15.5 #smallp/((gamma-1)smlrho)+.5(1)
> >>>  #1815.0 #smallp/((gamma-1)smlrho)+.5(vel_j)^2
> >>>
> >>> tiny            = 1.e-12
> >>>
> >>>
> >>> Best,
> >>>
> >>> Martin HE.
> >>> mh475 at cam.ac.uk <mailto:mh475 at cam.ac.uk>
> >>
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://flash.uchicago.edu/pipermail/flash-users/attachments/20090311/4ef11b7d/attachment.html 

From mh475 at cam.ac.uk  Wed Mar 11 09:56:22 2009
From: mh475 at cam.ac.uk (Martin Huarte-Espinosa)
Date: Wed, 11 Mar 2009 14:56:22 +0000
Subject: [FLASH-USERS] Flash3.1 output files permissions
In-Reply-To: <3182e9f70903110735w2517610fnbd1fa18db89e7da8@mail.gmail.com>
References: <3182e9f70903110735w2517610fnbd1fa18db89e7da8@mail.gmail.com>
Message-ID: <3182e9f70903110756rcb83898j1093de5f273789aa@mail.gmail.com>

Dear Flash community:

I'm having problems with the permissions on my Flash3.1 output files. When I
run flash without any cluster scheduler the chk files are produced with
permissions - rw - rw - - - -. But, if I send the job to a cluster using the
Condor scheduler I am not the owner of the files and thus I cannot open
them.

Any idea on how to solve/control this?

Thanks and good day.

Best,

Martin HE.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://flash.uchicago.edu/pipermail/flash-users/attachments/20090311/3e11d591/attachment.html 

From seyit at astro.rug.nl  Thu Mar 12 08:15:22 2009
From: seyit at astro.rug.nl (Seyit Hocuk)
Date: Thu, 12 Mar 2009 14:15:22 +0100
Subject: [FLASH-USERS] Possibility of restarting FLASH from a selected region
Message-ID: <49B90AEA.9020600@astro.rug.nl>

Hi Community,

I was wondering if it is possible to start a new Flash simulation from 
an end point of an older simulation by just selecting a part of the 
data, perhaps by a child block/level. This would now be the top block 
and act as the new initial conditions, forgetting everything else. I 
hope I am being clear in this. This way we could go much higher in 
resolution, without increasing the computational demands, by zooming in 
on an area and restarting simulation from there.

Thanks,
Seyit


From Kevin.M.Olson at drexel.edu  Thu Mar 12 08:24:14 2009
From: Kevin.M.Olson at drexel.edu (Kevin Olson)
Date: Thu, 12 Mar 2009 09:24:14 -0400
Subject: [FLASH-USERS] Possibility of restarting FLASH from a selected
	region
In-Reply-To: <49B90AEA.9020600@astro.rug.nl>
References: <49B90AEA.9020600@astro.rug.nl>
Message-ID: <434b205647a7005ac62c270482f894ba@drexel.edu>

Dear Seyit,

What version of FLASH are you using?  If you are using version 3.0 or 
above then you can refine up to 50 levels.

I don't think FLASH supports specifically what you are asking for.  You 
could try just keep the computational domain as it is, derefine in the 
regions where you are not interested and refine in the region where you 
are interested.  An advantage of doing this is that the original 
boundary conditions in your refined regions will be preserved.  A 
disadvantage is that the floating point computations may loose 
accuracy.

Best,
Kevin Olson

On Mar 12, 2009, at 9:15 AM, Seyit Hocuk wrote:

> Hi Community,
>
> I was wondering if it is possible to start a new Flash simulation from 
> an end point of an older simulation by just selecting a part of the 
> data, perhaps by a child block/level. This would now be the top block 
> and act as the new initial conditions, forgetting everything else. I 
> hope I am being clear in this. This way we could go much higher in 
> resolution, without increasing the computational demands, by zooming 
> in on an area and restarting simulation from there.
>
> Thanks,
> Seyit
>



From dubey at flash.uchicago.edu  Thu Mar 12 08:39:07 2009
From: dubey at flash.uchicago.edu (Anshu Dubey)
Date: Thu, 12 Mar 2009 08:39:07 -0500
Subject: [FLASH-USERS] Possibility of restarting FLASH from a selected
	region
In-Reply-To: <49B90AEA.9020600@astro.rug.nl>
References: <49B90AEA.9020600@astro.rug.nl>
Message-ID: <5b942a610903120639v52c52fa1k448ab44f33a0af73@mail.gmail.com>

This cannot be done now in the code as it is. But it is an interesting thought.

Kevin's suggestion is fairly easy, you can do derefinement based on
where you are in
the domain, the relevant files are
gr_markInRectangle/Ellipsoid/Radius, which you can
customize for your purpose.

If you really want to restart from a few blocks data, here is how you
could do it.
- Run the simulation as usual, and then write out the relevant blocks
into a separate file before
  exiting.  This could be a simple Fortran binary writeout.
- Write a simulation_initBlock where you read from the file.
-  Make a copy of  gr_expandDomain in your simulation directory, and
delete the part
   that repeatedly initializes simulation data to 0 and calls
Simulation_initBlock to fill it.
   You basically let paramesh populate the data in the blocks. You may
also want to
    keep the number of refinement steps at initialization relatively
low because this will be
   all interpolated data.

On Thu, Mar 12, 2009 at 8:15 AM, Seyit Hocuk <seyit at astro.rug.nl> wrote:
> Hi Community,
>
> I was wondering if it is possible to start a new Flash simulation from an
> end point of an older simulation by just selecting a part of the data,
> perhaps by a child block/level. This would now be the top block and act as
> the new initial conditions, forgetting everything else. I hope I am being
> clear in this. This way we could go much higher in resolution, without
> increasing the computational demands, by zooming in on an area and
> restarting simulation from there.
>
> Thanks,
> Seyit
>


From seyit at astro.rug.nl  Thu Mar 12 09:14:26 2009
From: seyit at astro.rug.nl (Seyit Hocuk)
Date: Thu, 12 Mar 2009 15:14:26 +0100
Subject: [FLASH-USERS] Possibility of restarting FLASH from a selected
 region
In-Reply-To: <434b205647a7005ac62c270482f894ba@drexel.edu>
References: <49B90AEA.9020600@astro.rug.nl>
	<434b205647a7005ac62c270482f894ba@drexel.edu>
Message-ID: <49B918C2.60002@astro.rug.nl>

Dear Kevin,

I am using FLASH 2.5 still. However, 50! levels? That's insane :).
Refining is especially interesting (my opinion) if you do structure/star 
formation where you have a core collapse and the density, temperature 
and pressure increases thereby reducing the timestep. In effect, for me, 
I can not go beyond level of ~10 refinement. However, starting from a 
smaller boxsize (in a specific region) would make it possible to do the 
same resolution (say 10) and result in a detailed image of that area. Of 
course, refining only this selected region and adapting the mesh from 
10-20, is more ideal (boundary conditions and stuff), it is much more 
computer demanding (even though the number of blocks do not seem 
increase much) and impossible to achieve for me.

I have the impression that other codes, like ENZO, can do this more easily.

Kind Regards,
Seyit



Kevin Olson wrote:
> Dear Seyit,
>
> What version of FLASH are you using?  If you are using version 3.0 or 
> above then you can refine up to 50 levels.
>
> I don't think FLASH supports specifically what you are asking for.  
> You could try just keep the computational domain as it is, derefine in 
> the regions where you are not interested and refine in the region 
> where you are interested.  An advantage of doing this is that the 
> original boundary conditions in your refined regions will be 
> preserved.  A disadvantage is that the floating point computations may 
> loose accuracy.
>
> Best,
> Kevin Olson
>
> On Mar 12, 2009, at 9:15 AM, Seyit Hocuk wrote:
>
>> Hi Community,
>>
>> I was wondering if it is possible to start a new Flash simulation 
>> from an end point of an older simulation by just selecting a part of 
>> the data, perhaps by a child block/level. This would now be the top 
>> block and act as the new initial conditions, forgetting everything 
>> else. I hope I am being clear in this. This way we could go much 
>> higher in resolution, without increasing the computational demands, 
>> by zooming in on an area and restarting simulation from there.
>>
>> Thanks,
>> Seyit
>>
>



From Kevin.M.Olson at drexel.edu  Thu Mar 12 11:25:20 2009
From: Kevin.M.Olson at drexel.edu (Kevin Olson)
Date: Thu, 12 Mar 2009 12:25:20 -0400
Subject: [FLASH-USERS] Possibility of restarting FLASH from a selected
	region
In-Reply-To: <49B918C2.60002@astro.rug.nl>
References: <49B90AEA.9020600@astro.rug.nl>
	<434b205647a7005ac62c270482f894ba@drexel.edu>
	<49B918C2.60002@astro.rug.nl>
Message-ID: <bf1295aa0adb58df4586074fda56cb72@drexel.edu>

Dear Seyit,

Since you are using FLASH2.5, what I suggested before will not work 
since the version of PARAMESH that is used does not allow more than 10 
refinement levels.  Newer versions PARAMESH allow more refinement 
levels and are only supported in FLASH 3.x.

I don't think doing what you want to do will help with regards to the 
timestep problem.
You would still need to take smaller timesteps in your 'zoomed' in 
region.

Also, I and others have found the almost all the time computing done in 
FLASH is done on the finest meshes.  So, derefining the mesh in 
uninteresting regions of your simulation and keeping them shouldn't be 
a severe performance hit at all.

I am not that familiar with the details of ENZO, but from what I have 
seen I think it is a good code and if it does what you need, give it a 
shot!

Best,
Kevin Olson

On Mar 12, 2009, at 10:14 AM, Seyit Hocuk wrote:

> Dear Kevin,
>
> I am using FLASH 2.5 still. However, 50! levels? That's insane :).
> Refining is especially interesting (my opinion) if you do 
> structure/star formation where you have a core collapse and the 
> density, temperature and pressure increases thereby reducing the 
> timestep. In effect, for me, I can not go beyond level of ~10 
> refinement. However, starting from a smaller boxsize (in a specific 
> region) would make it possible to do the same resolution (say 10) and 
> result in a detailed image of that area. Of course, refining only this 
> selected region and adapting the mesh from 10-20, is more ideal 
> (boundary conditions and stuff), it is much more computer demanding 
> (even though the number of blocks do not seem increase much) and 
> impossible to achieve for me.
>
> I have the impression that other codes, like ENZO, can do this more 
> easily.
>
> Kind Regards,
> Seyit
>
>
>
> Kevin Olson wrote:
>> Dear Seyit,
>>
>> What version of FLASH are you using?  If you are using version 3.0 or 
>> above then you can refine up to 50 levels.
>>
>> I don't think FLASH supports specifically what you are asking for.  
>> You could try just keep the computational domain as it is, derefine 
>> in the regions where you are not interested and refine in the region 
>> where you are interested.  An advantage of doing this is that the 
>> original boundary conditions in your refined regions will be 
>> preserved.  A disadvantage is that the floating point computations 
>> may loose accuracy.
>>
>> Best,
>> Kevin Olson
>>
>> On Mar 12, 2009, at 9:15 AM, Seyit Hocuk wrote:
>>
>>> Hi Community,
>>>
>>> I was wondering if it is possible to start a new Flash simulation 
>>> from an end point of an older simulation by just selecting a part of 
>>> the data, perhaps by a child block/level. This would now be the top 
>>> block and act as the new initial conditions, forgetting everything 
>>> else. I hope I am being clear in this. This way we could go much 
>>> higher in resolution, without increasing the computational demands, 
>>> by zooming in on an area and restarting simulation from there.
>>>
>>> Thanks,
>>> Seyit
>>>
>>
>



From seyit at astro.rug.nl  Fri Mar 13 05:21:57 2009
From: seyit at astro.rug.nl (Seyit Hocuk)
Date: Fri, 13 Mar 2009 11:21:57 +0100
Subject: [FLASH-USERS] Possibility of restarting FLASH from a selected
 region
In-Reply-To: <bf1295aa0adb58df4586074fda56cb72@drexel.edu>
References: <49B90AEA.9020600@astro.rug.nl>
	<434b205647a7005ac62c270482f894ba@drexel.edu>
	<49B918C2.60002@astro.rug.nl>
	<bf1295aa0adb58df4586074fda56cb72@drexel.edu>
Message-ID: <49BA33C5.4090100@astro.rug.nl>

Dear Kevin,

" I am not that familiar with the details of ENZO, but from what I have 
seen I think it is a good code and if it does what you need, give it a 
shot! "

Nor am I, but I will not switch to ENZO after spending so much time on 
FLASH. FLASH is in my opinion a better code for the stuff that I do. :)

Kind Regards,
Seyit


Kevin Olson wrote:
> Dear Seyit,
>
> Since you are using FLASH2.5, what I suggested before will not work 
> since the version of PARAMESH that is used does not allow more than 10 
> refinement levels.  Newer versions PARAMESH allow more refinement 
> levels and are only supported in FLASH 3.x.
>
> I don't think doing what you want to do will help with regards to the 
> timestep problem.
> You would still need to take smaller timesteps in your 'zoomed' in 
> region.
>
> Also, I and others have found the almost all the time computing done 
> in FLASH is done on the finest meshes.  So, derefining the mesh in 
> uninteresting regions of your simulation and keeping them shouldn't be 
> a severe performance hit at all.
>
> I am not that familiar with the details of ENZO, but from what I have 
> seen I think it is a good code and if it does what you need, give it a 
> shot!
>
> Best,
> Kevin Olson
>
> On Mar 12, 2009, at 10:14 AM, Seyit Hocuk wrote:
>
>> Dear Kevin,
>>
>> I am using FLASH 2.5 still. However, 50! levels? That's insane :).
>> Refining is especially interesting (my opinion) if you do 
>> structure/star formation where you have a core collapse and the 
>> density, temperature and pressure increases thereby reducing the 
>> timestep. In effect, for me, I can not go beyond level of ~10 
>> refinement. However, starting from a smaller boxsize (in a specific 
>> region) would make it possible to do the same resolution (say 10) and 
>> result in a detailed image of that area. Of course, refining only 
>> this selected region and adapting the mesh from 10-20, is more ideal 
>> (boundary conditions and stuff), it is much more computer demanding 
>> (even though the number of blocks do not seem increase much) and 
>> impossible to achieve for me.
>>
>> I have the impression that other codes, like ENZO, can do this more 
>> easily.
>>
>> Kind Regards,
>> Seyit
>>
>>
>>
>> Kevin Olson wrote:
>>> Dear Seyit,
>>>
>>> What version of FLASH are you using?  If you are using version 3.0 
>>> or above then you can refine up to 50 levels.
>>>
>>> I don't think FLASH supports specifically what you are asking for.  
>>> You could try just keep the computational domain as it is, derefine 
>>> in the regions where you are not interested and refine in the region 
>>> where you are interested.  An advantage of doing this is that the 
>>> original boundary conditions in your refined regions will be 
>>> preserved.  A disadvantage is that the floating point computations 
>>> may loose accuracy.
>>>
>>> Best,
>>> Kevin Olson
>>>
>>> On Mar 12, 2009, at 9:15 AM, Seyit Hocuk wrote:
>>>
>>>> Hi Community,
>>>>
>>>> I was wondering if it is possible to start a new Flash simulation 
>>>> from an end point of an older simulation by just selecting a part 
>>>> of the data, perhaps by a child block/level. This would now be the 
>>>> top block and act as the new initial conditions, forgetting 
>>>> everything else. I hope I am being clear in this. This way we could 
>>>> go much higher in resolution, without increasing the computational 
>>>> demands, by zooming in on an area and restarting simulation from 
>>>> there.
>>>>
>>>> Thanks,
>>>> Seyit
>>>>
>>>
>>
>



From ridg at cs.uchicago.edu  Tue Mar 17 02:56:12 2009
From: ridg at cs.uchicago.edu (ridg at cs.uchicago.edu)
Date: Tue, 17 Mar 2009 00:56:12 -0700
Subject: [FLASH-USERS] [SPAM] Returned mail: Data format error
Message-ID: <20090317102940.A5CA4282C03D@flash.uchicago.edu>

Your message was not delivered due to the following reason:

Your message was not delivered because the destination server was
unreachable within the allowed queue period. The amount of time
a message is queued before it is returned depends on local configura-
tion parameters.

Most likely there is a network problem that prevented delivery, but
it is also possible that the computer is turned off, or does not
have a mail system running right now.

Your message could not be delivered within 8 days:
Mail server 52.181.239.97 is not responding.

The following recipients did not receive this message:
<flash-users at flash.uchicago.edu>

Please reply to postmaster at flash.uchicago.edu
if you feel this message to be in error.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: transcript.bat
Type: application/octet-stream
Size: 28864 bytes
Desc: not available
Url : http://flash.uchicago.edu/pipermail/flash-users/attachments/20090317/bd553ebc/attachment.obj 

From latife at astro.rug.nl  Tue Mar 31 11:03:43 2009
From: latife at astro.rug.nl (M.A. Latife)
Date: Tue, 31 Mar 2009 18:03:43 +0200
Subject: [FLASH-USERS] writing data in parallel run
Message-ID: <49D23EDF.2080902@astro.rug.nl>

Hi,
Dear All,
I have very simple question, i am writing data to file in cool_block.f90 
subroutine. I am running job on parallel machine (using MPI). Every time 
amount of data written to file is changed but it is never written 
completely. How can i write complete data to file? It has to do 
something with MPI.
Regards
Latif