Building SAMRAI on OS X Tiger

Originally posted by Dan on 05:00 Mon 29 May 2006, last modified 13:41 Mon 7 May 2007.

File under: 3rd party tools c++ os x programming

I actually started this post around early April, but discovered that my SAMRAI install wasn't actually quite right; which took me a month or so to sort out. Eventually I got SAMRAI and a Level Set Method library working. Only now have I found time to putting some information on the web that some people might find useful.

These notes apply to installing SAMRAI on a PowerPC Mac. I have started trying to install it on an Intel Mac, but haven't had time to resolve some of the issues. If I ever get round to doing it, then I'll make another post.

Okay, so one might well ask, "Why am I blogging about this inane topic?" and well, perhaps a better question would be, "Why am I building SAMRAI v2.1.0?" Or even, "What on earth is it?" All good questions, which I'll address in due course; for now, I'll assume that you have your own answers, and therefore might be interested in my own experience in (eventually) getting SAMRAI working on OS X Tiger, with gcc 4.0.1, g77 and xlf8.1.

Here follows the steps I suggest taking to build a working distribution of SAMRAI...

Install the latest Developer Tools from Apple.
You can download them from here: http://developer.apple.com/tools/download/ or use your Tiger DVD. Note that Xcode 2.3 was released in May 2006, so if you obtained Tiger before that date, download the updated Developer Tools from the above link.
Obtain a Fortran compiler.
I used the g77 compiler as distributed on http://hpc.sourceforge.net. You might also be using IBM's xlf compiler.
Download some source files:
- SAMRAI: http://www.llnl.gov/casc/SAMRAI/register/get_info.html
- Mpich1: http://www-unix.mcs.anl.gov/mpi/mpich1/download.html
- HDF5: http://hdf.ncsa.uiuc.edu/register5.html
Set up your environment variables.
- If you don't have one, create a bash profile in your home folder.
  maeby:~ dt05r$ touch ~/.bash_profile
  maeby:~ dt05r$ open ~/.bash_profile
- In here we need to set up the $PATH environment variable. My bash profile looks somthing like this, to give you an idea. There are some other variables that will be required later, such as RSHCOMMAND which will need to be set.
  export PATH=$PATH:/usr/local/bin
  export PATH=$PATH:/sw/bin
  export PATH=$PATH:/usr/local/mpich/bin
  export PATH=$PATH:/usr/local/samrai/bin
  export PATH=$PATH:/usr/local/visit/bin
  export PATH=$PATH:/opt/ibmcmp/xlf/8.1/bin
  export RSHCOMMAND=ssh
  export TERM=xterm-color
- If you intend to use ssh to connect remotely to your machine, then you'll also want to copy the entire contents of your bash_profile to bashrc.
  maeby:~ dt05r$ cp ~/.bash_profile ~/.bashrc
  I also advise that you set your machines up to enable password-less ssh, and use a good ssh-agent tool, such as SSH Agent. This will enable you to test your environment variables from a remote machine. Check out this IBM tutorial for more information.
Installing Mpich 1
- Don't install mpich2, as SAMRAI isn't compatible with it on OS X.
- There is a binary for mpich1 on hpc.sourceforge.net, however I recommend not using it, and instead building it directly from source, it doesn't take that long, and it means you know exactly where things are going.
- In the mpich source directory, execute:
  maeby:~/Desktop/mpich-1.2.7p1 dt05r$ ./configure --prefix=/usr/local/mpich
  Keep an eye on the output for any problems, especially with regard to g77. Consult the config.log if necessary.
- After it's configured, execute
  maeby:~/Desktop/mpich-1.2.7p1 dt05r$ make
  maeby:~/Desktop/mpich-1.2.7p1 dt05r$ make install
  Add, /usr/local/mpich/bin to your path as described above.
- If you have problems during the make stage, where the linker cannot seem to find certain symbols, in particular "_sprintf$LDBLStub" and "_fprintf$LDBLStub", then a quick suggestion would be to revert to gcc-3.3 and make mpich using that.
  maeby:~ dt05r$ sudo gcc_select 3.3
  Password:
  Default compiler has been set to: gcc version 3.3 20030304 (Apple Computer, Inc. build 1819)
  maeby:~ dt05r$
  If your primary concern is using SAMRAI in a serial mode then this shouldn't present much of a problem later down the line. If you want to really use SAMRAI on a cluster, then you might want to get mpich1 build using gcc-4.0. Let me know if you get it working!
- A quick sanity check of mpich1 would be...
  
  You might notice here that my two machines, Tobias and Maeby have got mpich built using gcc-4.0.1 and gcc-3.3 respectively. So, it is possible to get mpich built on OS X Tiger using gcc-4.0.1, the problem is I can't quite remember exactly how I managed it on Tobias!
Installing HDF5. This should be fairly easy...
maeby:~/Desktop/hdf5-1.6.5 dt05r$ ./configure --prefix=/usr/local/hdf5
maeby:~/Desktop/hdf5-1.6.5 dt05r$ [...]
maeby:~/Desktop/hdf5-1.6.5 dt05r$ make
maeby:~/Desktop/hdf5-1.6.5 dt05r$ sudo make install
Configuring SAMRAI
- We need to configure the folder structure of SAMRAI's build location. Do the following:
  maeby:~/Desktop dt05r$ mkdir SAMRAI-v2.1.0
  maeby:~/Desktop dt05r$ tar -zxf SAMRAI-v2.1.0.gtar.Z -C ~/Desktop/SAMRAI-v2.1.0/
  maeby:~/Desktop dt05r$ cd SAMRAI-v2.1.0/
  maeby:~/Desktop/SAMRAI-v2.1.0 dt05r$ mkdir macosx
  maeby:~/Desktop/SAMRAI-v2.1.0 dt05r$ ls
  SAMRAI macosx
  maeby:~/Desktop/SAMRAI-v2.1.0 dt05r$ cd macosx
  maeby:~/Desktop/SAMRAI-v2.1.0/macosx dt05r$
- Run the configure script:
  maeby:~/Desktop/SAMRAI-v2.1.0/macosx dt05r$ sh ../SAMRAI/configure --prefix=/usr/local/samrai --enable-opt=-O3 --with-x --with-hdf5=/usr/local/hdf5
Editing SAMRAI's source files.
We have to edit some source files to make sure they compile on the Mac. Most of these are detailed in the config/mac folder, and on Kevin Chu's webpage. However I'll also detail them here for brevity. These are all references to the ~/Desktop/SAMRAI-v2.1.0/SAMRAI/ folder that you should now have.
- include/tbox/MPI.h
  We need to use the full path name to mpich's mpi.h file. Otherwise we reference the MPI.h header due to case insensitivity. So starting at line 16 we want:
```
#ifdef HAVE_MPI
#ifndef included_mpi
#define included_mpi
#include "/usr/local/mpich/include/mpi.h"
#endif
#endif
```
- source/toolbox/memory/MemoryUtilities.C
  Here there are two things we need to change, firstly the location of malloc.h
```
#include <stdio.h>
#include <stdlib.h>
#include <malloc/malloc.h>
```
  We also need to put a preprocessor check around the printMemoryInfo function, which starts around line 60.
```
#ifdef HAVE_TAU
	void MemoryUtilities::printMemoryInfo(ostream& os) 
	{
		.
		.
		.
	}
#endif
```
- source/toolbox/base/IEEE.C and
  source/toolbox/templates/special/MathUtilitiesSpecial.C
  Here we need to use a preprocessor check to handle the isnan function. For IEEE.C this starts on line 260.
```
#if defined(__GNUC__)&&(__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 2))
   int i = __isnan(f);
#else
   int i = isnan(f);
#endif		
```
```
#if defined(__GNUC__)&&(__GNUC__ > 3 || (__GNUC__ == 3 && __GNUC_MINOR__ >= 2))
   int i = __isnand(d);
#else
   int i = isnan(d);
#endif		
```
  You'll need to change this slightly between the two files, as they have different arguments. Note that in the configure notes for Mac (in the SAMRAI config directory), there is an error in that it uses the float function (__isnan(f)) instead of the double function (__isnand(d)) for the second edit. So watch out for that. Essentially the code should now look like this, for IEEE.C and MathUtilitiesSpecial.C respectively.
- source/toolbox/templates/special/StringSpecial.C
  This is mentioned in relation to a possible problem with hdf5, I think it's still relevant. You need to add a preprocessor check around the explicit string template instantiation. This is right at the bottom of the file:
```
#if defined(__GNUC__) && (__GNUC__ == 3 && __GNUC_MINOR__ >= 2)
#if defined(__ppc__) && !defined(HAVE_HDF5)
// Templates for complex functions
template std::basic_string<char, std::char_traits<char>, std::allocator<char> >
	std::operator+<char, std::char_traits<char>, std::allocator<char> >
	(std::basic_string<char, std::char_traits<char>, std::allocator<char> >
	const&, std::basic_string<char, std::char_traits<char>,
		std::allocator<char>> const&);
#endif
#endif	
```
  Please just copy the #if lines, and not the template line, as I might have made a typo making that HTML compatible.
- source/apputils/plotting/VisItDataWriter.C and
  include/VisItDataWriter.C
  You are probably going to be using VisIt to render the results of your SAMRAI efforts, in which case the following change also needs to be made. This isn't mentioned in the config/mac notes, but was pointed out to me by Kevin, who was informed by someone called Brian Taylor. On line 4222 you need to change H5T_NATIVE_CHAR to H5T_NATIVE_INT.
```
errf = H5Tinsert(s1_tid, 
	   "data_is_defined",
	   HOFFSET(patchMinMaxStruct, patch_data_on_disk),
	   H5T_NATIVE_INT);
```
  Remember to make the changes to both files - this is very important, as the include file is just copied to your system.
Correcting SAMRAI's Makefile.config
The Makefile.config that SAMRAI creates, will almost certainly need some tweaking. The changes made will differ depending on which compilers you are using. I've only used the gcc-4.0.1 C compiler; for Fortran sources I've used g77 and xlf. Note that the Makefile.config to edit is ~/Desktop/SAMRAI-v2.1.0/macos/config/Makefile.config.
Essentially the lines we are going to edit are the $(LDLIBS) and $(CXXLDLIBS) variables, which are actually the same. Firstly, copy them, and then comment out one of each with a # character. Then generate something like this, I've done this twice for the two Fortran compilers.
- xlf
  Note these variables should all be one one line each, but due to presentation I've spread it over a few.
```
LDFLAGS = -L$(OBJECT)/lib -L/usr/local/hdf5/lib 
-L/usr/local/mpich/lib -L/usr/lib/gcc/powerpc-apple-darwin8/4.0.1 
-L/usr/lib/gcc/powerpc-apple-darwin8/4.0.1/../../.. -L/usr/X11R6/lib 
$(LDFLAGS_EXTRA)

LDLIBS = $(LDLIBS_EXTRA) -bind_at_load -lhdf5 -lmpich -lpmpich -lpthread -lmx 
-lxlfmath -lSM -lICE -lX11 -L/opt/ibmcmp/lib -L/opt/ibmcmp/xlf/8.1/lib
-L/opt/ibmcmp/xlsmp/1.3/lib -lxlf90 -lm
```
  The $(CXXLD_FLAGS) and $(CXXLDLIBS) are the same, so you can use copy them. Obviously if the location of your libraries differs from mine, then make the appropriate changes.
- g77
```
LDFLAGS = -L$(OBJECT)/lib -L/usr/local/hdf5/lib -L/usr/local/mpich/lib 
-L/usr/X11R6/lib -L/usr/lib $(LDFLAGS_EXTRA)

LDLIBS = $(LDLIBS_EXTRA) -bind_at_load -lhdf5 -lmpich -lpmpich -lpthread -lz
-lSM -lICE -lX11 -lg2c -lm
```
  Again, the $(CXXLD_FLAGS) and $(CXXLDLIBS) are the same, so you can use copy them.
I'm actually fairly sure that you don't need the paths to the gcc compiler libs, as they're probably found automatically, but it shouldn't hurt to have them in there.
Make SAMRAI
We can now make SAMRAI:
```
maeby:~/Desktop/SAMRAI-v2.1.0/macosx dt05r$ make
```
This should take a couple of hours. If it stops then something's gone wrong, and either I've forgotton to mention something, or you've done something wrong, in which case email me (dt05r <at> ecs . soton . ac . uk).
Testing SAMRAI
Before you test SAMRAI, it's necessary to rebuild the libraries using ranlib.

Then you can make and execute some of the examples.
Installing SAMRAI
To install SAMRAI, we need to make one last edit, this time to the Makefile in the macosx folder. Change the install rule, to install-samrai for example, on line 235.
```
install-samrai:
	$(INSTALL) -d -m 755 $(INSTDIR)/config
	$(INSTALL) -d -m 755 $(INSTDIR)/lib		
	.
	.
```
Then just run sudo make install-samrai.
```
maeby:~/Desktop/SAMRAI-v2.1.0/macosx dt05r$ sudo make install-samrai
```
Rebuld the SAMRAI libs
Finally, you need to rebuild the SAMRAI libraries using ranlib.
```
maeby:~/Desktop/SAMRAI-v2.1.0/macosx dt05r$ sudo ranlib -s /usr/local/samrai/lib/*.a
```