1. smithyformula(5)
  2. smithyformula(5)

NAME

smithyformula - writing formulas for smithy

DESCRIPTION

The main goal of formulas is to consolidate all knowledge required to build a software package. This can include:

Once written it's easy to see everything required to build a given piece of software. Reproducing those steps is as simple as running one command but only if the formula is as complete as possible.

It's common for build scripts to run the compilation but omit patches, changes to makefile, or any other modification to the source. With formulas it's easy to make patches and output any files needed to compile software.

CREATING NEW FORMULAS

The best way to create a new formula is to start with a working example. There are many complete working examples in the smithy_formulas repo.

If you want to create a new formula from scratch you can use the smithy formula new subcommand. For more info on this command run smithy help formula new. To create a new formula file you need to know the homepage and a url to download the file. To create a new formula for subversion you might run:

smithy formula new \
  --name=subversion \
  --homepage=http://subversion.apache.org/ \
  http://mirror.cogentco.com/pub/apache/subversion/subversion-1.7.8.tar.bz2

The format of the new sub-command is smithy formula new [command options] URL. The options and arguments are:

--name

This is the name used for the formula file and class, if omitted smithy will try to guess the name based on the download URL

--homepage

This should be the main homepage for the software

URL

A download URL for the software, this argument is required but may be "none" if you plan to checkout the code through a version control system or copy from another location as part of the formula

This will create a formula file inside ~/.smithy/formulas or the first formula directory specified in the $SMITHY_CONFIG file. In either case, the full path to that file will be displayed.

STRUCTURE

Formulas attempt to create a domain specific language with the support of a full programming language, ruby. The structure of a formula is the same as a ruby class. For example:

class SubversionFormula < Formula

end

Every method call that defines the formula will happen between these two lines.

FORMULA FILE AND CLASS NAMING

Formulas follow a specifc naming scheme. The filename should end in '_formula.rb' and start with the name of the software in all lowercase characters. The class name should be the same name specified in the file but CamelCased and end in 'Formula'.

RUBY BASICS

We will cover most of the basics you need for formula writing here but if you would like more info on ruby you might read through Ruby in Twenty Minutes or try another source on the Ruby Documentation page.

FORMULA DSL METHODS

These methods should be defined at the highest level of the formula file, right after the class GitFormula < Formula line.

homepage

REQUIRED - Defines the homepage, e.g. "http://git-scm.com/"

url

REQUIRED - The full URL to download a package, e.g. "http://git-core.googlecode.com/files/git-1.8.3.4.tar.gz" may also be "none"

sha1,sha256,md5

A hash of the downloaded file to verify the download performed correctly, e.g. sha1 "fe633d02f7d964842d7ea804278b75120fc60c11"

version

Manually define the version number, if omitted smithy will guess the version number from the url. This works best when the filename in a url follows the name-version.tar... format. If a url does not follow that format then manually specifying the version with the version method works best.

supported_build_names

This method takes any number of strings or regexes. When used smithy will check the supplied build_name and validate it against what is specified under supported_build_names. If the build_name does not match a string or regex the install is aborted.

This is useful for when a formula must have specific build names in order to install correctly. For example, using the this line in a formula:

supported_build_names "python2.7", "python3"

And running smithy formula install python_nose/1.3.4/python2.6 (note python2.6 does not match anything in the supported_build_names line above). Then smithy will output the following abort the install:

==> PythonNoseFormula supported build names:
  python2.7
  python3
==> ERROR: use a build_name that includes any of the above patterns

When using strings with supported_build_names smithy will check to see if the build_name contains any supplied substrings.

If using regexes they must simply match the build name. Here is an example using regexes:

supported_build_names /python.*gnu/

That regex checks for python followed by any number of characters followed by gnu. When running smithy formula install python_numpy/1.9.2/python2.7.9 with the above you should see:

==> PythonNumpyFormula supported build names:
  python.*gnu
==> ERROR: use a build_name that includes any of the above patterns

params

This method is used to specify some unchanging variables you would like to use in any block throughout the formula. It can be useful if you want some variables located at the top of the formula file that are easily changed. It takes a hash where the keys are variable names and the values are the variable contents. For example:

class ExampleFormula do
  homepage "http://example.com"
  url "http://example.com/example-1.2.3.tar.gz"

  params example_api_version: "1.0",
         build_folder: "example1.0"

  def install
    system "./configure --api-version #{example_api_version} --build-dir #{build_folder}"
  end
end

concern for_version("1.2.3") do

Concerns are used allow formulas to support multiple versions. When used, their contents will override the same methods defined at the root level of a formula class. They are a part of the ActiveSupport Concerning Module

To specify a concern for version 1.2.3 of a package add the following to a formula (replacing the comment with the methods you would like to override).

concern for_version("1.2.3") do
  included do
    # override methods here
  end
end

Anything formula DSL method can be overridden including def install.

Here is an example of a python formula that supports version 2.7.9 and 3.4.3. The concern only overrides the download urls and md5s.

class PythonFormula < Formula
  homepage "www.python.org/"

  depends_on "sqlite"

  module_commands ["unload python"]

  concern for_version("2.7.9") do
    included do
      url "https://www.python.org/ftp/python/2.7.9/Python-2.7.9.tgz"
      md5 "5eebcaa0030dc4061156d3429657fb83"
    end
  end

  concern for_version("3.4.3") do
    included do
      url "https://www.python.org/ftp/python/3.4.3/Python-3.4.3.tgz"
      md5 "4281ff86778db65892c05151d5de738d"
    end
  end

  def install
    module_list
    ENV["CPPFLAGS"] = "-I#{sqlite.prefix}/include"
    ENV["LDFLAGS"]  = "-L#{sqlite.prefix}/lib"
    system "./configure --prefix=#{prefix} --enable-shared"
    system "make"
    system "make install"
  end
end

Using the above formula it is possible to install python with the following and smithy will download the correct tarball for each version:

smithy formula install python/2.7.9/sles11.3_gnu4.3.4
smithy formula install python/3.4.3/sles11.3_gnu4.3.4

disable_group_writable

Calling this method within the formula will skip setting group writable file permissions after the build is complete. It's equivalent to running smithy with the --disable-group-writable option. See smithy help for more info on global command line options.

depends_on

This method expects either a single string or an array of strings that define dependencies for this formula. e.g.

depends_on "curl"
depends_on [ "cmake", "qt", "openssl", "sqlite" ]
depends_on %w{ cmake qt openssl sqlite }

Using this method ensures that if a given dependency is not met smithy will abort the installation. It also provides a way to query dependent packages information within the install method later on. For example if you write depends_on "curl" in your formula you gain access to an object named curl inside the install method. This allows you to do things like:

system "./configure --prefix=#{prefix} --with-curl=#{curl.prefix}"

In the above example #{curl.prefix} is an example of a ruby interpolated string, everything between the #{ } is ruby code. curl.prefix will return a string with the location curl is installed in.

The strings passed to depends_on are just the locations of installed software. If you required a specific version of a dependency you could use specify the version or build numbers of existing installed software. e.g.

depends_on [ "cmake/2.8.11.2/sles11.1_gnu4.3.4", "qt/4.8.5", "sqlite" ]

Assuming your software root is /sw/xk6 smithy would look for the above software installs in /sw/xk6/cmake/2.8.11.2/sles11.1_gnu4.3.4 /sw/xk6/qt/4.8.5/* and /sw/xk6/sqlite/*/*. The * works similar to shell globbing. If you needed to install a python module that depends on a specific version of another python module you might use:

depends_on [ "python/3.3.0", "python_numpy/1.7.1/*python3.3.0*" ]

This would require a given formula to have access to both /sw/xk6/python/3.3.0/* and a python module with a build name that includes python3.3.0 located at /sw/x6/python_numpy/1.7.1/*python3.3.0*

You will also probably need to specifiy dependencies conditionally upon the type of build you are performing. It's recommended to add the type of build to the build name when installing. Given that, you can key off build names to specify dependencies. Taking the python example further, lets extend it to support multiple versions of python. You can pass a ruby block to the depends_on method to make it more dynamic. The syntax for this is:

depends_on do
  ...
end

Any ruby code may go in here the last executed line of the block should be an array of strings containting the dependencies. Lets use a ruby case statement for this:

depends_on do
  case build_name
  when /python3.3/
    [ "python/3.3.0", "python_numpy/1.7.1/*python3.3.0*" ]
  when /python2.7/
    [ "python/2.7.3", "python_numpy/1.7.1/*python2.7.3*" ]
  end
end

In this example case statement switches on the build_name. The when /python3.3/ will be true if the build_name contains the python3.3. The /python3.3/ syntax is a regular expression.

This allows the formula to set it's dependencies based off the type of build thats being performed. Lets say this formula is python_matplotlib. You could run either of these commands to install it and expect the dependencies to be set correctly:

smithy formula install python_matplotlib/1.2.3/python3.3.0
smithy formula install python_matplotlib/1.2.3/python2.7.3

module_commands

This method defines the module commands that must be run before system calls within the def install part of the modulefile. It expects an array of strings with each string being a module command. e.g.

module_commands [ "load szip", "load hdf5" ]

A more complicated example:

module_commands [
  "unload PE-gnu PE-pgi PE-intel PE-cray",
  "load PE-gnu",
  "load cmake/2.8.11.2",
  "load git",
  "swap gcc gcc/4.7.1",
  "swap ompi ompi/1.6.3"
]

module_commands also accepts ruby blocks the syntax for this is:

module_commands do
  ...
end

This can be used to dynamically set which modules to load based on the build_name. Here is an example that loads the correct python version:

module_commands do
  commands = [ "unload python" ]

  case build_name
  when /python3.3/
    commands << "load python/3.3.0"
  when /python2.7/
    commands << "load python/2.7.3"
  end

  commands << "load python_numpy"
  commands << "load szip"
  commands << "load hdf5/1.8.8"
  commands
end

This block starts by creating a variable named commands as an array with a single item "unload python". Next a case statement is used to determine which version of python we are compiling for. commands << "load python/3.3.0" will append "load python/3.3.0" to the end of the array. See the ruby documentation on the Array Class method for more info on the << operator. After that, it appends a few more modules to load. The last line of the block must be the array itself so that when the block is evaluated by smithy, it recieves the expected value.

Assuming this is a formula for python_h5py running smithy formula install python_h5py/2.1.3/python3.3 results in an array containing: [ "unload python", "load python/3.3.0", "load python_numpy", "load szip", "load hdf5/1.8.8" ]

modules

This command is similar to the module_commands method. It accepts an array of strings with the names of modulefiles that must be loaded. It's different from module_commands in that it expects only names of modules and loads them in the order specified. Additionally it runs a module purge to unload all modules before loading the passed array of modules. e.g.

modules [ "java" ]

This line would run module purge and module load java before any system command. modules also accepts ruby blocks similar to module_commands and depends_on. Here is an example using ruby blocks:

modules do

mods = [ ]
case build_name
when /gnu/
  mods << "PrgEnv-gnu"
when /pgi/
  mods << "PrgEnv-pgi"
when /intel/
  mods << "PrgEnv-intel"
when /cray/
  mods << "PrgEnv-cray"
end
mods

end

This block would result in the formula running module purge followed by module load PrgEnv-gnu if the build_name contains gnu and similarly for pgi, intel, and cray.

modulefile

This method expects the a string that represents the modulefile. Generally modulefiles in smithy take two forms ones that point to a single build and ones that use multiple builds and set the build based on a users environment (already loaded modules). It's recommended to have one modulefile per application version and set multiple builds dynamically inside the modulefile.

Writing modulefiles is a topic in and of itself. For details on the modulefile format see the modulefile(4) manpage Modulefiles are written in tcl and can take many forms.

Here is an example of a modulefile that points to a single build. It's convenient to use heredoc string quoting in ruby so that the string can span multiple lines. e.g.

modulefile <<-MODULEFILE.strip_heredoc
  #%Module
  proc ModulesHelp { } {
     puts stderr "<%= @package.name %> <%= @package.version %>"
     puts stderr ""
  }
  module-whatis "<%= @package.name %> <%= @package.version %>"

  set PREFIX <%= @package.prefix %>

  prepend-path PATH            $PREFIX/bin
  prepend-path LD_LIBRARY_PATH $PREFIX/lib
  prepend-path MANPATH         $PREFIX/share/man
MODULEFILE

The <<-MODULEFILE.strip_heredoc syntax denotes the beginning for a multi-line string. The string ends with MODULEFILE. You can substitute any word for MODULEFILE. The .strip_heredoc method removes leading whitespace from the string so the output isn't unecessarily indented.

The modulefile definition uses the erb format Anything between the <%= ... %> delimiters will be interpreted as ruby code. There are a few helper methods that you can use inside these delimiters see the next section titled [MODULEFILE HELPER METHODS][] for details.

A more complicated modulefile may examine already loaded modules to determine which build to load. For instance if the user has gcc or a gnu programming environment module loaded then your modulefile will want to load the gnu build. Here is an example designed to dynamically set the build:

#%Module
proc ModulesHelp { } {
   puts stderr "<%= @package.name %> <%= @package.version %>"
   puts stderr ""
}
# One line description
module-whatis "<%= @package.name %> <%= @package.version %>"

<% if @builds.size > 1 %>
<%= module_build_list @package, @builds %>

set PREFIX <%= @package.version_directory %>/$BUILD
<% else %>
set PREFIX <%= @package.prefix %>
<% end %>

# Helpful ENV Vars
setenv <%= @package.name.upcase %>_DIR $PREFIX
setenv <%= @package.name.upcase %>_LIB "-L$PREFIX/lib"
setenv <%= @package.name.upcase %>_INC "-I$PREFIX/include"

# Common Paths
prepend-path PATH            $PREFIX/bin
prepend-path LD_LIBRARY_PATH $PREFIX/lib
prepend-path MANPATH         $PREFIX/share/man
prepend-path INFOPATH        $PREFIX/info
prepend-path PKG_CONFIG_PATH $PREFIX/lib/pkgconfig
prepend-path PYTHONPATH      $PREFIX/lib/python2.7/site-packages
prepend-path PERL5PATH       $PREFIX/lib/perl5/site_perl

The main difference from the first example is the <%= if @builds.size > 1 %> block. This basically checks to see if we have installed multiple builds or not. If that condition is true everything up until the <% else %> will be put in the modulefile. Otherwise, if we have only one build, set PREFIX <%= @package.prefix %> will be put in the modulefile.

def install

REQUIRED - This is the method that runs the software installation process. It normally runs system commands, performs patches, and sets environment variables. e.g.

def install
  system "./configure"
  system "make"
  system "make install"
end

The contents of the install method depends heavily on the software being installed. For a list of additional helper methods for use inside install see the [FORMULA HELPER METHODS][] section.

additional_software_roots

This method takes an array of strings (or a block that returns an array of strings) containing paths to additional software root directories. This is useful for when you have a package that must be installed to two or more separate locations. The source is still extracted to one location and the compilation is performed there. The def install method is just executed again with the additional prefixes. For example say our formula contained the line:

additional_software_roots ["/lustre/atlas/sw"]

Then when we run smithy formula install python_numpy/1.9.2/python2.7.9 then python will still be installed to the default prefix of /sw/xk6/python_numpy/1.9.2/python2.7.9 It will also get installed to /lustre/atlas/sw/xk6/python_numpy/1.9.2/python2.7.9

You can combine this with config_value to generalize you formulas further. Say we always need to know where software gets installed on the lustre filesystem. We can add this line to the config file: lustre-filesystem: /lustre/atlas/sw and in out formulas say:

additional_software_roots [ config_value("lustre-filesystem") ]

FORMULA HELPER METHODS

These methods are designed to be used within the def install method of a formula file or within a block passed to one [FORMULA DSL METHODS][].

name

This will return the name of the application being installed. It is the same as the APPLICATION part of the smithy formula install APPLICATION/VERSION/BUILD command. It can be used as a variable as well as inside of a string using the #{ ... } delimiters. e.g. "#{name}"

version

Similar to the above, this returns the version number.

build_name

Same as the name and version methods, this will return the build name of the applcation.

prefix

This line will return the full prefix to an application. If we run smithy formula install bzip2/1.0.4/pgi13.4 and our software-root is /sw/xk6 this command will return /sw/xk6/bzip2/1.0.4/pgi13.4

hostname

This method will return the hostname of the machine smithy is running on.

arch

This method will return the current architecture specified in the config file for the current hostname. For example if the config file contains:

---
software-root: /sw
file-group-name: ccsstaff
hostname-architectures:
  titan-ext: xk6
  titan-login: xk6

And you run smithy on titan-ext1 then arch will return xk6

cray_linux_version

If smithy is running on a cray system this method will return the Cray Linux Environment version. Otherwise it will return false.

cray_system?

This method will return true if smithy is running on a cray system.

config_value

This method lets you retrieve arbitrary values from the $SMITHY_CONFIG file. Since that file is in YAML format you can easily append extra values for use in your formulas.

For example, if your system has a specific path for a filesystem your formula needs to know about then you can add it you the config file. Say we add a line titled lustre-filesystem:

---
software-root: /sw
file-group-name: ccsstaff
hostname-architectures:
  titan-ext: xk6
  titan-login: xk6
lustre-filesystem: /lustre/atlas/sw

You can then call config_value("lustre-filesystem") in your formulas and get /lustre/atlas/sw back.

system

This method accepts a string or multiple strings separated by commas. It will run the given command in a subshell and setup the modules as defined by the module_commands or modules methods. Each call to system is independent from the last. Modules are reloaded and environment variables are reset.

system_python

The system_python method runs python using a system command and sets the PYTHONPATH environment variable to a location in the current prefix. This is useful for running a setup.py for a python module. It is designed to be used like:

system_python "setup.py install --prefix=#{prefix} --compile"

Under the hood it is running:

system "PYTHONPATH=$PYTHONPATH:#{prefix}/lib/#{python_libdir(current_python_version)}/site-packages python",
  "setup.py install --prefix=#{prefix} --compile"

Where python_libdir(current_python_version) is the lib directory for the version of python in the $PATH e.g. python2.7 or python3.4

python_module_from_build_name

This is a helper to change a full python version specified in the build_name e.g. python2.7.9 or python3.4.3 to a version with a forward slash e.g. python/2.7.9 or python/3.4.3 respectively.

It is useful for when a formula is installed with a python version in the build_name and you want to get the corresponding python version string for use in a module_commands or depends_on block. For example:

depends_on do
  python_module_from_build_name
end

module_commands do
  [ "unload python",
    "load #{python_module_from_build_name}" ]
end

If the above is used when installing a formula with python2.7.9 in the build name e.g. smithy formula install python_numpy/1.9.2/python2.7.9_gnu4.8.2 then python_module_from_build_name will return python/2.7.9.

python_version_from_build_name

Like python_module_from_build_name, this is a helper to extract the python version from the build_name. It is useful when a formula is installed with a python version in the build_name and you want to get the corresponding python version string for use in a depends_on block. For example:

depends_on "python_numpy/1.9.2/#{python_version_from_build_name}*"

If the above is used when installing a formula with python2.7.9 in the build_name e.g. smithy formula install python_scipy/0.15.1/python2.7.9_gnu4.8.2 then python_version_from_build_name will return python2.7.9.

python_libdir(version)

This method will extract the major and minor version numbers for the python version being installed. This is only useful for the python formula. It is useful when referencing the python libdir, which is shared among different patch versions. When installing python/3.4.3/sles11.1_gnu4.8.2 the result of python_libdir(version) will be python3.4.

module_list

This method will run module list and print it's output durring the install process. Useful for verifying the modules loaded are the ones you want.

module_is_available?

This method will check if a given modulename is available on the system you are performing the installation on. It takes one string argument, the module name. It can be used within the def install, depends_on, module_commands, or modules methods.

if module_is_available?("hdf5/1.8.8")
  ...
end

module_environment_variable

Using this method will return the contents of an environment varible set by a modulefile. It takes two string arguments, the module name and the environment variable name. For example, if you wished to get the value of the $HDF5_DIR variable set within the hdf5/1.8.8 module you could run:

hdf5_prefix = module_environment_variable("hdf5/1.8.8", "HDF5_DIR")

Using this method to get environment variable set by modules is necessary since the modules are only set before running a system command. See Setting Environment Variables for more info. It can be used within the def install, depends_on, module_commands, or modules methods.

By combining the [module_is_available?][] and module_environment_variable methods you can conditionally retrieve the contents of environment variables set within a given module.

if module_is_available?("hdf5/1.8.8")
  hdf5_prefix = module_environment_variable("hdf5/1.8.8", "HDF5_DIR")
end

patch

The patch method is a convinience method to apply patches to code. Behind the scenes it creates a file named patch.diff with the passed content and runs patch -p1 <patch.diff. Using the heredoc syntax works best to strip leading whitespace. For example:

patch <<-EOF.strip_heredoc
  diff --git a/CMake/cdat_modules/cairo_external.cmake b/CMake/cdat_modules/cairo_external.cmake
  index e867fb2..22fb40c 100644
  --- a/CMake/cdat_modules/cairo_external.cmake
  +++ b/CMake/cdat_modules/cairo_external.cmake
  @@ -1,7 +1,7 @@

   set(Cairo_source "${CMAKE_CURRENT_BINARY_DIR}/build/Cairo")
   set(Cairo_install "${cdat_EXTERNALS}")
  -set(Cairo_conf_args --disable-static)
  +set(Cairo_conf_args --enable-gobject=no --disable-static)

   ExternalProject_Add(Cairo
     DOWNLOAD_DIR ${CDAT_PACKAGE_CACHE_DIR}
EOF

Any input you provide must be compatible with the patch command. You can use interpolated strings #{ ... } to modify the content of patches as well:

patch <<-EOF.strip_heredoc
  diff --git a/Makefile.in b/Makefile.in
  new file mode 100644
  index 0000000..1235d4b
  --- /dev/null
  +++ b/Makefile.in
  @@ -0,0 +1,12 @@
  +SHELL = /bin/sh
  +PLAT = LINUX
  +BLLIB = #{acml_prefix}/gfortran64/lib/libacml.a
  +CBLIB = #{prefix}/lib/libcblas.a
  +CC = gcc
  +FC = gfortran
  +LOADER = $(FC)
  +CFLAGS = -O3 -DADD_
  +FFLAGS = -O3
  +ARCH = ar
  +ARCHFLAGS = r
  +RANLIB = ranlib
EOF

PYTHON SPECIFIC HELPERS

Python module installations require some extra consideration. To make life a bit easier Smithy provides the following methods.

MODULEFILE HELPER METHODS

<%= @package.name %>

This will return the name of the application being installed. It is the same as the APPLICATION part of the smithy formula install APPLICATION/VERSION/BUILD command.

<%= @package.version %>

Similar to the above, this returns the version number.

<%= @package.build_name %>

Same as the name and version methods, this will return the build name of the applcation.

<%= @package.prefix %>

This line will return the full prefix to an application. If we run smithy formula install bzip2/1.0.4/pgi13.4 and our software-root is /sw/xk6 this command will return /sw/xk6/bzip2/1.0.4/pgi13.4

<%= @builds %>

The @builds variable is an array of strings that contain the list of available builds for a given application. Say we have a bzip2 formula and ran the following installs:

smithy formula install bzip2/1.0.4/gnu4.3.4
smithy formula install bzip2/1.0.4/gnu4.7.2
smithy formula install bzip2/1.0.4/pgi13.4
smithy formula install bzip2/1.0.4/intel12

The directory structure for the above builds would look like (assuming /sw/xk6 is the software-root):

/sw/xk6/bzip2/1.0.4
`--- modulefile
|  `--- bzip2
|     `--- 1.0.4
`--- gnu4.3.4
|  `--- bin
|  `--- include
|  `--- lib
|  `--- source
|  `--- share
`--- gnu4.7.2
|  `--- bin
|  `--- include
|  `--- lib
|  `--- source
|  `--- share
`--- pgi13.4
|  `--- bin
|  `--- include
|  `--- lib
|  `--- source
|  `--- share
`--- intel12
   `--- bin
   `--- include
   `--- lib
   `--- source
   `--- share

The @builds array would then be [ "gnu4.3.4", "gnu4.7.2", "pgi13.4", "intel12" ]. This lets you figure out what builds exist and use them in your modulefile.

<%= @builds.size %>

size is a standard ruby method that counts the number of elements in an array. For the above example this would return 4.

<%= module_build_list @package, @builds %>

This is a helper method that will generate the tcl necessary to conditionally load builds based on what compiler programming environment modules a user has loaded. It takes @package and @builds as arguments. Using the above bzip2 example the result of using this method would be:

if [ is-loaded PrgEnv-gnu ] {
  if [ is-loaded gcc/4.3.4 ] {
    set BUILD gnu4.3.4
  } elseif [ is-loaded gcc/4.7.2 ] {
    set BUILD gnu4.7.2
  } else {
    set BUILD gnu4.7.2
  }
} elseif [ is-loaded PrgEnv-pgi ] {
  set BUILD pgi13.4
} elseif [ is-loaded PrgEnv-intel ] {
  set BUILD intel12
} elseif [ is-loaded PrgEnv-cray ] {
  puts stderr "Not implemented for the cray compiler"
}
if {![info exists BUILD]} {
  puts stderr "[module-info name] is only available for the following environments:"
  puts stderr "gnu4.3.4"
  puts stderr "gnu4.7.2"
  puts stderr "intel12"
  puts stderr "pgi13.4"
  break
}

<%= python_module_build_list @package, @builds %>

This method is similar to the previously mentioned [module_build_list][] except that it will create a block specific to python versions. It requires that a full python version be part of the current build name e.g. python2.7.9. It will look at the builds installed for the current formula and create a corresponding tcl if block. It will also check to see if python modules exist that correspond to the build names.

For example, lets assume we perform the following installs for the python_numpy formula:

smithy formula install python_numpy/1.9.2/python2.7.9_gnu4.8.2
smithy formula install python_numpy/1.9.2/python3.4.3_gnu4.8.2

Then the following builds will exist:

python_numpy
`--- 1.9.2
   `--- python2.7.9_gnu4.8.2
   `--- python3.4.3_gnu4.8.2

If the python/2.7.9 and python/3.4.3 modules exist then [python_module_build_list][] method will create the following output:

if [ is-loaded python/2.7.9 ] {
  set BUILD python2.7.9_gnu4.8.2
  set LIBDIR python2.7
} elseif [ is-loaded python/3.4.3 ] {
  set BUILD python3.4.3_gnu4.8.2
  set LIBDIR python3.4
}
if {![info exists BUILD]} {
  puts stderr "[module-info name] is only available for the following environments:"
  puts stderr "python/2.7.9"
  puts stderr "python/3.4.3"
  break
}

It is designed to be used like this:

<%= python_module_build_list @package, @builds %>
set PREFIX <%= @package.version_directory %>/$BUILD

<% if ... %>

This is standard erb ruby code. Delimiters like <% ... %> do NOT put their results in the final modulefile, they are only used for control flow. Delimiters with the extra = sign <%= ... %> will put their results in the final modulefile.

This is best used to conditionally render content to the modulefile and takes the form:

<% if @builds.size > 1 %>
  ...
<% else %>
  ...
<% end %>

Where @builds.size > 1 can be any expression which returns true or false. If the if condition is true then the lines between the if and else will be put in the modulefile, otherwise lines between the else and end will be used.

COMMON OPERATIONS

Change Working Directory

Changing the working directory accomplished by the Dir.chdir method. It takes one argument as a string. It's best to always work from the prefix of the installation. You can concatenate strings using a + sign.

Dir.chdir prefix
Dir.chdir prefix+"/source"

Running Shell Commands

In ruby you can execute any shell command using backtick delimiters. Commands run using this method will NOT load any required modulefiles. It will however return the standard output as a string

`ln -svf file1 file2`
results = `ln -svf file1 file2`

If you need modulefiles loaded use the system command instead.

system "ln -svf file1 file2"

There are many ruby methods available that are the equivalent of running the shell counterparts. See the these pages for more info:

Setting Environment Variables

Ruby provides the ENV hash for accessing and setting environment variables. Here is an example of setting environment variables:

ENV["CC"]  = "gcc"
ENV["CXX"] = "g++"
ENV["F77"] = "gfortran"
ENV["F90"] = "gfortran"
ENV["FC"]  = "gfortran"

And getting their values back:

cppflags = ENV["CPPFLAGS"]

This works with one caveat, you cannot access or modify variables set by modules loaded by the formula. Modules are loaded and reset before each system command.

If you need to access the contents of an evironment variable set by a module use the module_environment_variable helper method.

If you need to change the contents of an environment variable set by a loaded module you will need to make it part of the system command. For example, say you loaded the netcdf module and needed to change the $NETCDF_DIR variable. You could run any of these:

system "NETCDF_DIR=/opt/cray/netcdf/4.2.0/generic ./configure"
system "export NETCDF_DIR=/opt/cray/netcdf/4.2.0/generic ;",
  "./configure"
system "export NETCDF_DIR=/opt/cray/netcdf/4.2.0/generic
  ./configure"

Creating Files

Using ruby to create files is simple. With this and the #{ ... } delimiters you can add dynamic content to the files you write. The basic syntax for writing files is:

File.open("path/to/file", "w+") do |file|
  file.write "..."
end

This syntax uses a ruby block with an argument. File.open("...", "w+") passes the file handle to the block as a variable named file. Between the do |file| and end lines is the block. When ruby reaches the end of the block the file is closed.

The "w+" argument tells ruby to open the file with read-write and truncate the existing file to zero length or create a new file for reading and writing. See this page for more info on the different modes.

Here is a good example from the mpi4py formula

File.open("mpi.cfg", "w+") do |f|
  f.write <<-EOF.strip_heredoc
    [cray]
    mpi_dir = /opt/cray/mpt/5.6.3/gni/mpich2-gnu/47
    mpicc   = cc
    mpicxx  = CC
  EOF
end

This will create a new file named "mpi.cfg" in the current working directory. If you wanted to make it a bit more dynamic you might wish to set the contents based on an environment variable like this:

mpidir = module_environment_variable("cray-mpich2", "CRAY_MPICH2_DIR")
File.open("mpi.cfg", "w+") do |f|
  f.write <<-EOF.strip_heredoc
    [cray]
    mpi_dir = #{mpidir}
    mpicc   = cc
    mpicxx  = CC
  EOF
end

Putting it Together

You can combine these methods in many ways. This bit of code is from the uvcdat formula and it creates symlinks from an openssl installation into a directory under the uvcdat prefix.

Dir.chdir prefix
openssl_files = %w{
  include/openssl
  lib/pkgconfig/libcrypto.pc
  lib/pkgconfig/libssl.pc
  lib/pkgconfig/openssl.pc
  lib/engines
  lib/libcrypto.a
  lib/libcrypto.so
  lib/libcrypto.so.1.0.0
  lib/libssl.a
  lib/libssl.so
  lib/libssl.so.1.0.0
}
FileUtils.mkdir_p "Externals/include"
FileUtils.mkdir_p "Externals/lib/pkgconfig"
openssl_files.each do |file|
  system "ln -sf #{openssl.prefix}/#{file} #{prefix}/Externals/#{file}"
end

It begins by changing the working directory to the installation prefix. Then, creates an array of strings named openssl_files containing relative paths to files needing to be symlinked. It then creates directories that might not exist yet using FileUtils.mkdir_p. Then it iterates through the openssl_files array and runs one system command per array element. That system command uses an openssl.prefix method that is made available by the depends_on defined earlier in the formula to get the location of the openssl installation.

SEE ALSO

smithy(1)

  1. March 2015
  2. smithyformula(5)