-- EumedUser - 2010-02-17

WP3 Applications Support WIKI Home

Proxy renewal using MyProxy

After some pain and confusion, the official recipe for automatic proxy renewal using a MyProxy server was found to be this one.

EUMEDGRID-Support Software

The EUMEDGRID-Support project offers a set of software packages aiming to help mediterranean scientific communities to explore and/or directly exploit Grid computing capabilities. The selection of the software has been done taking in consideration the most important Virtual Researcher Communities (VRCs) developed in the context of the European Grid Infrastructure (EGI). It is possible to keep track of the installed software thanks to the following pages:

The actual software selection foresee two different kind of categories. The first set of packages consists of general purpose software with no specific relationship with a particular scientific domain. Inside this category of software there are: Compilers, Interpreters, Data visualization and scientific frameworks. The second set of packages consists of well known software products targeted to a specific scientific domain.

EUMEDGRID-Support maintains also a list of applications developed and maintained by the mediterranean community. This software is published by the WP3 application portal.

There exists also a set of packages considered as mandatory in order to be considered 'certified' for the eumed infrastructure. These mandatory packages are described here.

SOFTWARE LIST

Many Grid Infrastructures rely on well know software products serving lot of different experiments in different research domains ranging form particle physics, High Energy Physics, Biomedicine, Chemistry, Earth Sciences and many other disciplines. These generic applications are then of crucial importance in attracting scientific user communities who may base their own experiments on top of these already available applications. EUMEDGRID-Support project selected a group of applications covering the most important scientific domains with particular attention to the existing research groups that are currently dealing with project partners. The list of selected application is splitted in two separate categories of software products. A firts list contains very general purpose applications like compilers, frameworks, interpreters, etc. The second list contains domain specific software. The list of selected applications is available below:

General purpose software

Name Type
Root Development Framework
Intel compilers Development suite
Octave Math interpreter
GCC4.5 Development suite
GCC auxiliary libraries gmp-4.3.2 mpfr-2.4.2 mpc-0.8.2: required by GCC 4.5
FFTW library fftw: a C subroutine library for computing the discrete Fourier transform (DFT)
SCILAB Math

Specific software

Name Domain
GATE Biomed
ASTRA Cultural Heritage
Blast Biomed
ClustalW Biomed
GAMESS Computational Chemistry
GROMACS Computational chemistry
HMMER Life Sciences
MrBayes Life Sciences
NAMD Computational Chemistry
OpenFOAM Engineering
Rosetta Life Sciences
WENMR Biomed
WRF4G Earth Science

Getting the SoftwareManager role

Users who have been granted the special SoftwareManager role should, before installing software at GRID sites, create a special, privileged proxy by running the command

  voms-proxy-init --voms eumed:/eumed/Role=SoftwareManager

USING THE grid-swinstall PACKAGE

All packages are installed using a simplified procedure offered by the grid-swinstall package, see at the bottom of the page for the latest available version of grid-swinstall_<date>.tgz tarfile.

The grid-swinstall package is maintained by Fulvio Galeazzi any software manager are free (and welcome) to provide changes to the current scripts and/or add new installation/validation procedures; in such a case please keep updated Fulvio Galeazzi about the changes in order to incorporate changes in forthcoming versions of the package.

Overview of the grid-swinstall package

The distributed tarfile contains a script GridManageSW which is a generic procedure performing a lot of useful actions. Generally speaking, to install a software package you will, in general, need to:

  • download a tarfile from the GRID and untar it: this step is called installation
  • compile the package (if need be): this step is called validation

For each package, you will thus need to launch the script twice, once for "installation" and, as soon as installation has successfully completed, a second time for validation. Keep reading, for a detailed example.

The script takes care of publishing intermediate tags (read next chapter) to signal pending installation activity, and the successful or problematic completion of each step. If a software package is in status "processing" and you submit again another installation/validation job, this will exit immediately: this ensures that steps are performed one at a time. The script will also recursively fix ownership and privileges for each file/directory such that all eumed users may access those files, and all different sgmeumed users have write access (this is needed because over time the same person may be mapped to two different sgmeumed accounts). It also produces a quite verbose log of what is being done, so that in case of troubles it should be rather easy to understand what has gone wrong.

Tags format being published by grid-swinstall package

The tags which are published when using grid-swinstall have a format, explained in detail below, which has a direct correspondance to the disk path under which the software will be found. For example, a package tagged VO-eumed-prod-pkg1-verX-gccany will be found at the path $VO_EUMED_SW_DIR/prod/pkg1/verX/gccany/: as you see, we have stripped the VO-eumed part, and replaced each further - with /, it's so simple.

Package VO-eumed-prod-local provides the script getrelpath.sh which does the work for you: you supply a tag name and it returns the relevant subpath under $VO_EUMED_SW_DIR:

[galeazzi@ui2-4 Grid]$ $VO_EUMED_SW_DIR/prod/local/bin/getrelpath.sh VO-eumed-prod-pkg1-verX-gccany
/prod/pkg1/verX/gccany

The general format for tags is the following VO-<VO_name>-<status>-<pkgName>-<pkgVersion>-<SOname>-<SOarch>-<compilerName>, like for example VO-eumed-prod-gromacs-v040503-SL5-x86_64-gccany, where:

  • <VO_name>: the name of the VO, eumed most probably since you are reading this
  • <status>: this can be prod or test, the former for official production packages, the latter for packages to be used for test purposes only
  • <pkgName>: the name of the package
  • <pkgVersion>: the version of the package
  • <SOname>: the name of the operating system the package is supposed to run on, typically it will be SL5
  • <SOarch>: the architecture for which the package was compiled, typically it will be i386 or x86_64
  • <compilerName>: the compiler used. If the default compiler found was used, the string will be gccany

Also note that the tag name may contain the string ANY: any string after ANY is intended as a comment, and will not be interpreted by the string-to-pathname converter getrelpath.sh.

While software installation is ongoing, you will see some other tags being published. Assuming you are installing VO-eumed-pkg1-verX, you will see:

VO-eumed-pkg1-verX-processing
whenever a script is running (installation, validation, uninstallation)
VO-eumed-pkg1-verX-aborted-<action>
if the <action> script failed (installation, validation)
VO-eumed-pkg1-verX-to-be-validated
after successful installation of the package. As the tag suggests, you are now ready to run the validation script.
VO-eumed-pkg1-verX
after successful running of the validation script

Last important note: after successful installation, GridManageSW will also touch a hidden file .<software-tag>, for example:

[galeazzi@ui2-4 ~]$ $VO_EUMED_SW_DIR/prod/local/bin/getrelpath.sh VO-eumed-prod-gamess-v101001-SL5-x86_64-gccany
/prod/gamess/v101001/SL5/x86_64/gccany
[galeazzi@ui2-4 ~]$ ls -al $VO_EUMED_SW_DIR/prod/gamess/v101001/SL5/x86_64/gccany | tail -1
-rw-rw-r--  1 sgmeumed001 sgmeumed      0 Apr  5 17:37 .VO-eumed-prod-gamess-v101001-SL5-x86_64-gccany

Installation instructions

If you want to install package pkg1, make sure:

  • a tarfile is available and registered in the catalogue under /grid/eumed/SoftRepo/ (I use LFC catalogue lfc.ulakbim.gov.tr): the tarfile name is referenced in the .jdl scripts, see option --tar
  • you have files needed for installation and validation: install_pkg1.jdl, validate_pkg1.jdl, and possibly also validate_pkg1.sh: check in the .jdl scripts the tarfile name, and the tag name which the scripts are supposed to publish
  • run glite-wms-job-submit -a -o install_pkg1.id -r <theCEwhereYouWantToInstall> install_pkg1.jdl this will download the tarfile and explode it in the proper place
  • after successful job completion, run: glite-wms-job-submit -a -o install_pkg1.id -r <theCEwhereYouWantToInstall> validate_pkg1.jdl this will (possibly) run compilation of the package (also using validate_pkg1.sh script) and publish the tag if everything went fine

If the validation job ran correctly, the software tag should have been published by the site.

Package versions and dependencies

Software packages may depend on each other. Below is a table showing the latest tags which are published by the current version of grid-swinstall, and some notes illustrating inter-dependencies.

Package name Tag Notes
local VO-eumed-prod-local-ANY-20110124 This is the first package which needs to be installed. Anything else depends on it.
fftw VO-eumed-prod-fftw-v030202-SL5-x86_64-gccany FFTW 3.2.2. Pre-requisite for Gromacs. Configuration switches:--enable-float --enable-shared --enable-threads
gccauxlib VO-eumed-prod-gccauxlib-v020000-SL5-x86_64-gccany Pre-requisite for gcc. Tag version v020000 is just my convention. Contains gmp-4.3.2 mpfr-2.4.2 mpc-0.8.2.
gcc VO-eumed-prod-gcc-v040502-SL5-x86_64-gccany GCC 4.5.2. Depends on gccauxlib. Configuration switches: --enable-languages=c,c++,fortran --exec-prefix=$finalDir --prefix=$finalDir --enable-shared --enable-threads=posix --disable-checking --disable-libgcj --enable-__cxa_atexit --with-system-zlib --disable-libunwind-exceptions --with-gmp=${theSWdir}$gccauxdir --with-gmp-include=${theSWdir}$gccauxdir/include --with-gmp-lib=${theSWdir}$gccauxdir/lib --with-mpfr=${theSWdir}$gccauxdir --with-mpfr-include=${theSWdir}$gccauxdir/include --with-mpfr-lib=${theSWdir}$gccauxdir/lib --with-mpc=${theSWdir}$gccauxdir --with-mpc-include=${theSWdir}$gccauxdir/include --with-mpc-lib=${theSWdir}$gccauxdir/lib
gamess VO-eumed-prod-gamess-v101001-SL5-x86_64-gccany GAMESS 20101001R1. config file modifed to run unattended, see patch file in .tar.gz. Compiled with arch=linux64 gfortran Atlas=none sockets.
gate VO-eumed-prod-gate-v060000-SL5-x86_64-gccany GATE 6.0.0. Pre-compiled: for further info contact Giuseppe
gromacs VO-eumed-prod-gromacs-v040503-SL5-x86_64-gccany Gromacs 4.5.3. Depends on fftw. Configuration switches: -with-fft=fftw3
namd VO-eumed-prod-namd-v020802-SL5-x86_64-multicore NAMD 2.8.2. Compiled from NAMD_2.8b1_Linux-x86_64-multicore.tar.gz
mrbayes VO-eumed-prod-mrbayes-v030102-SL5-x86_64-gccany MrBayes 3.1.2, based on source file from http://mrbayes.csit.fsu.edu/. Our tarfile also includes readline source files (compiled during validation step). Compiled with: CPATH=./include/ LIBRARY_PATH=./lib/ ARCHITECTURE=unix MPI=yes
root VO-eumed-prod-root-v052800b-SL5-x86_64-gccany ROOT version 5.28. Compiled from sources with configuration switches: ./configure linux --prefix=${finalRoot} --incdir=${finalRoot}/include --libdir=${finalRoot}/lib --etcdir=${finalRoot}/etc --datadir=${finalRoot} --docdir=${finalRoot} --cintincdir=${finalRoot}/cint --enable-rpath --disable-globus --enable-soversion --enable-table --enable-shared --enable-minuit2 --enable-roofit --enable-explicitlink --enable-mathcore --enable-mathmore --enable-builtin_pcre --enable-builtin-freetype --disable-python --disable-xrootd --disable-xml
scilab VO-eumed-prod-scilab-v050301-SL5-x86_64-gccany SciLab 5.3.1. Pre-compiled for Linux x86_64.

ADVANCED USE: preparing your own installation scripts

The philosophy of grid-swinstall is rather simple. You can read the GridManageSW script as well as the various install_*.jdl and validate_*.jdl for further information, but I will present here a full example.

Let's assume you have:

  • a tarfile myPkg-v000001.tar.gz
  • a software tag you'd like to publish: VO-eumed-test-myPkg-v000001
  • corresponding to the tag name above, our software will be installed under $VO_EUMED_SW_DIR/test/myPkg/v000001/. This imposes some constraint on the content of the tarfile:
    • if you are installing pre-compiled binaries, you need to make sure that the tarfile does not contain a common first-level subdirectory. I mean, people will expect to find the application under path $VO_EUMED_SW_DIR/test/myPkg/v000001/ so typically the binaries will be under $VO_EUMED_SW_DIR/test/myPkg/v000001/bin/: hence, if the tarfile has a structure similar to:
      [galeazzi@ui2-4 Packages]$ tar tzvf scilab-5.3.1.bin.linux-x86_64.tar.gz | head
      drwxr-xr-x scilab/scilab     0 2011-03-03 17:19:22 scilab-5.3.1/
      -rw-r--r-- scilab/scilab  8099 2011-03-03 17:19:22 scilab-5.3.1/CHANGES_4.X
      -rw-r--r-- scilab/scilab  1633 2011-03-03 17:19:22 scilab-5.3.1/README_Unix
      -rw-r--r-- scilab/scilab 30661 2011-03-03 17:19:22 scilab-5.3.1/CHANGES_5.1.X
      -rw-r--r-- scilab/scilab  1907 2011-03-03 17:19:22 scilab-5.3.1/RELEASE_NOTES_5.1.X
      -rw-r--r-- scilab/scilab  4854 2011-03-03 17:19:22 scilab-5.3.1/RELEASE_NOTES_5.0.X
      drwxr-xr-x scilab/scilab     0 2011-03-03 17:19:22 scilab-5.3.1/bin/
      -rwxr-xr-x scilab/scilab 413344 2011-03-03 17:18:42 scilab-5.3.1/bin/XML2Modelica
      -rwxr-xr-x scilab/scilab  30706 2011-03-03 17:19:11 scilab-5.3.1/bin/scilab
      lrwxrwxrwx scilab/scilab      0 2011-03-03 17:19:22 scilab-5.3.1/bin/scilab-adv-cli -> scilab
      [galeazzi@ui2-4 Packages]$ tar tzvf scilab-5.3.1 | head
      scilab-5.3.1.bin.linux-x86_64.tar.gz  scilab-5.3.1-x86_64.tar.gz
      you need to re-package it such that it looks like:
      [galeazzi@ui2-4 Packages]$ tar tzvf scilab-5.3.1-x86_64.tar.gz | head
      drwxr-xr-x galeazzi/grid     0 2011-03-03 17:19:22 ./
      -rw-r--r-- galeazzi/grid 28189 2011-03-03 17:19:22 ./CHANGES_5.0.X
      -rw-r--r-- galeazzi/grid 21097 2011-03-03 17:19:22 ./COPYING
      -rw-r--r-- galeazzi/grid  7372 2011-03-03 17:19:22 ./ACKNOWLEDGEMENTS
      -rw-r--r-- galeazzi/grid  2337 2011-03-03 17:19:22 ./RELEASE_NOTES_5.2.X
      drwxr-xr-x galeazzi/grid     0 2011-03-03 17:19:11 ./share/
      drwxr-xr-x galeazzi/grid     0 2011-03-03 17:19:12 ./share/locale/
      drwxr-xr-x galeazzi/grid     0 2011-03-03 17:19:11 ./share/locale/ca_ES/
      drwxr-xr-x galeazzi/grid     0 2011-03-03 17:19:11 ./share/locale/ca_ES/LC_MESSAGES/
      -rw-r--r-- galeazzi/grid 461101 2011-03-03 17:19:11 ./share/locale/ca_ES/LC_MESSAGES/scilab.mo
    • if you are compiling from source code, you can adjust paths in the validation stage (read below). Note however a very important thing: for obvious reasons, the validation stage needs to be run unattended. Hence, you may have to apply some tricks and patch the original configuration scripts (see for example the GAMESS tarfile, containing a patch for the configure script).

Once you have your tarfile ready, you need to copy to the GRID, bearing in mind:

  • the catalogue to use is LFC_HOST = lfc.ulakbim.gov.tr
  • the GridManageSW scripts assumes that tarfiles will be found under /grid/eumed/SoftRepo/
  • for the sake of redundancy, it is advisable to copy the tarfile to at least two different SE

Find below the list of commands I am typically using: in this example, first copy is registered at GARR, second copy at DZ-01-ARN.

[galeazzi@ui2-4 Packages]$ lcg-cr -d gridsrv3-4.dir.garr.it -l lfn:/grid/eumed/SoftRepo/myPkg-v000001.tar.gz file:myPkg-v000001.tar.gz
guid:53f6a7f1-e08c-418e-a907-bdb3e6199078
[galeazzi@ui2-4 Packages]$ lcg-lr lfn:/grid/eumed/SoftRepo/myPkg-v000001.tar.gz
srm://gridsrv3-4.dir.garr.it/dpm/dir.garr.it/home/eumed/generated/2011-04-20/file01e269b4-af1c-4886-a0de-29d51fc68454
[galeazzi@ui2-4 Packages]$ lcg-rep --vo eumed -n 4 -d se01.grid.arn.dz -v -P /grid/eumed/SoftRepo/myPkg-v000001.tar.gz srm://gridsrv3-4.dir.garr.it/dpm/dir.garr.it/home/eumed/generated/2011-04-20/file01e269b4-af1c-4886-a0de-29d51fc68454
Using grid catalog type: UNKNOWN
Using grid catalog : lfc.ulakbim.gov.tr
VO name: eumed
Checksum type: None
Trying SURL srm://gridsrv3-4.dir.garr.it/dpm/dir.garr.it/home/eumed/generated/2011-04-20/file01e269b4-af1c-4886-a0de-29d51fc68454 ...
Source SE type: SRMv2
Source SRM Request Token: d97c8978-3dbf-480e-b0dc-327faf522eb2
Destination SE type: SRMv2
Destination SRM Request Token: 9c03aa1b-7d33-485e-a666-69a808556421
Source URL: srm://gridsrv3-4.dir.garr.it/dpm/dir.garr.it/home/eumed/generated/2011-04-20/file01e269b4-af1c-4886-a0de-29d51fc68454
File size: 1902051
Destination specified: se01.grid.arn.dz
Source URL for copy: gsiftp://gridsrv3-4.dir.garr.it/gridsrv3-4.dir.garr.it:/data/eumed/2011-04-20/file01e269b4-af1c-4886-a0de-29d51fc68454.58471.0
Destination URL: gsiftp://se01.grid.arn.dz/se01.grid.arn.dz:/data/eumed/2011-04-20/myPkg-v000001.tar.gz.2008.0
# streams: 1
      1902051 bytes     72.84 KB/sec avg    130.94 KB/sec inst
Transfer took 27070 ms
Using LFN: lfn:(null)
Using GUID: guid:53f6a7f1-e08c-418e-a907-bdb3e6199078
Registering SURL: srm://se01.grid.arn.dz/dpm/grid.arn.dz/home/eumed//grid/eumed/SoftRepo/myPkg-v000001.tar.gz (53f6a7f1-e08c-418e-a907-bdb3e6199078)
Destination URL registered in file catalog: srm://se01.grid.arn.dz/dpm/grid.arn.dz/home/eumed//grid/eumed/SoftRepo/myPkg-v000001.tar.gz
[galeazzi@ui2-4 Packages]$ lcg-lr lfn:/grid/eumed/SoftRepo/myPkg-v000001.tar.gzsrm://gridsrv3-4.dir.garr.it/dpm/dir.garr.it/home/eumed/generated/2011-04-20/file01e269b4-af1c-4886-a0de-29d51fc68454
srm://se01.grid.arn.dz/dpm/grid.arn.dz/home/eumed//grid/eumed/SoftRepo/myPkg-v000001.tar.gz

You are now ready for the installation phase. By installation I mean the following actions:

  • downloading of the package from the GRID
  • creation of the destination directory
  • un-tar of the archive
  • recursively change ownership and access rights (add SGMgroup write-ability)
  • publication of the software tag -to-be-validated

All the above tasks are accomplished by the GridManageSW script alone, you just need to submit one job like:

glite-wms-job-submit -a -e https://wms-4.dir.garr.it:7443/glite_wms_wmproxy_server -r ce-cr-02.ts.infn.it:8443/cream-lsf-grid -o install_myPkg.id install_myPkg.jdl

where install_myPkg.jdl looks like:

Executable = "GridManageSW";
InputSandbox = {"GridManageSW"};
OutputSandbox = {"stdout"};
stdoutput = "stdout";
stderror = "stdout";
Environment = { "theVO=eumed", "LCG_GFAL_INFOSYS=bdii.eumedgrid.eu:2170" , "LFC_HOST=lfc.ulakbim.gov.tr" };
Arguments = "--install --tar myPkg-v000001.tar.gz  --tag VO-eumed-test-myPkg-v000001";

namely, you are submitting GridManageSW passing as arguments:

  • --install: this means, obviously, you are going to install something
  • --tar : the name of the tarfile to be downloaded (from /grid/eumed/SoftRepo/, remember?)
  • --tag : the tag you will eventually publish</verbatim>

After the installation has started at the remote site, the CE will publish tag myPkg-v000001-processing. After some time, you may check the output of the installation job and the tags being published by the CE: if installation went OK the site should publish: myPkg-v000001-to-be-validated otherwise it will publish myPkg-v000001-aborted-installation (or in some exceptional cases, if things were really bad, the job will have completed and the site will still publish the processing tag).

Let's assume we can proceed with the validation phase. For this step you will also need a validation script which should do whatever is needed to get a fully working software: hence the validation script should perform both configuration and compilation, or just nothing (in the case of pre-compiled packages). However, it is mandatory to prepare a script which, in case of success, prints a line containing SUCCESS: the presence of such line is used by GridManageSW to understand which tag should be published: VO-eumed-test-myPkg-v000001 in case of success, or VO-eumed-test-myPkg-v000001-aborted-validation otherwise.

For the validation you would submit a validate_myPkg.jdl which may look like:

Executable = "GridManageSW";
InputSandbox = {"GridManageSW", "validate_dummy.sh"};
OutputSandbox = {"stdout"};
stdoutput = "stdout";
stderror = "stdout";
Environment = { "theVO=eumed", "LCG_GFAL_INFOSYS=bdii.eumedgrid.eu:2170" };
Arguments = "--validate --tag VO-eumed-test-myPkg-v000001 --validate_script validate_dummy.sh";

Of course we are using as arguments:

  • --validate: do I really need to say more about this?
  • --validate_script : which is the script you want to execute. The validate_dummy.sh script is copied below, and as you can verify it's really dummy and does nothing, apart from printing the SUCCESS line:
    #!/bin/bash
    #
    # $Id: validate_dummy.sh 7 2011-03-16 14:33:45Z galeazzi $
    #
    echo "This is a dummy script, which only returns success"
    echo "=== SUCCESS "
    

For more complicated setup, please take a look at the validate_root.jdl and accompanying validate_root.sh scripts. I paste here the .jdl

:

Executable = "GridManageSW";
InputSandbox = {"GridManageSW", "validate_root.sh"};
OutputSandbox = {"stdout"};
stdoutput = "stdout";
stderror = "stdout";
Environment = { "theVO=eumed", "LCG_GFAL_INFOSYS=bdii.eumedgrid.eu:2170", "LFC_HOST=lfc.ulakbim.gov.tr" };
#  validate_script_arg is name of link which will be placed under ./prod/root/ hence it MUST be different from 5th field in tag name VO-argo-prod-root-<something_else>-<rest_of_tag_name>
Arguments = "--validate --tag VO-eumed-prod-root-v052800b-SL5-x86_64-gccany --validate_script validate_root.sh --validate_script_arg 5.28.00b";

Note in particular the argument --validate_script_arg which is an argument (for a list, enclose it in "ticks") to be passed to the --validate_script.

Note that in the validation script you may use some variables which are defined by the calling script GridManageSW, namely:

  • ${theSWdir}: is set to the base software directory for the VO, $VO_EUMED_SW_DIR/ in our example
  • ${sw_dir}: is set to the destination directory for the software package, $VO_EUMED_SW_DIR/test/myPkg/v000001/ in our example

You are welcome to play with grid-swinstall. Please don't forget to contact Fulvio so that your updates may be inserted in future versions of the package. Enjoy!

Intel Compilers Installation

This software cannot be installed with the standard procedure of sending a job submission with SoftwareManager role.

The installation of this software must be performed by each site administrator executing instructions below on all site CEs.

Pay Attention Some special features of this software that use Event-based Sampling (EBS) can be installed only if CE' CPU consists of a genuine Intel® Architecture, and on a non-virtual OS.

Getting the software

  1. Go to the Intel' web page Non-Commercial Software Download, read it, and click  Accept at the bottom left;
  2. You now are in the Software Development Products* page where all products available are listed;
  3. Click "on Intel® Parallel Studio XE 2011 for Linux*";
  4. In the new page which appears enter your email address, your location and click on  Continue ;
  5. In the new page wich appears, select first the compiler you want to download Select product: Product for 32-bit/x86-64bit development (1 GB - Download Manager use recommended for this file) and then the delivery mechanism (web/download manager);
  6. You will receive an email with a serial number; keep it!;
  7. Unpack the archives containing the compilers and check that the directory does not have any spaces in the name;
  8. Execute the script ./install.sh and follow the instructions; when requested, insert the serial number; if the are missing dependencies, solve them; one dependency that may appear is that for libstdc++.so.0.5; to solve it, install compat-libstdc++-33 from using yum; install the version compatible with the number of bits of you Operating System: 32 or 64;

Before you install, important notes

*NOTE:* you can make just a single installation for the whole site (so any VO may benefit from your effort).

  • Execute the installation script passing the --SHARED_INSTALL option
  • Remember to specify a shared destination path: for example, you may install under $VO_SW_DIR/local/intel/Compiler/XE2011
  • At the end of the installation, create the directory $VO_SW_DIR/local/intel/Compiler/XE2011/composerxe-2011.4.191/licenses/ and copy therein the license file:
         cp -p /opt/intel/licenses/*lic $VO_SW_DIR/local/intel/Compiler/XE2011/composerxe-2011.4.191/licenses/
  • create a script similar to the following one, to set environment variables in a proper way: replace $VO_SW_DIR as appropriate for your site. I assume such script will be called $VO_SW_DIR/local/intel/Compiler/setupIntelCompiler.sh and I also assume I have installed two versions of the Intel Compiler, namely 11.1 and IntelXE-2011 (as subdirectories 11.1 and XE2011, respectively, under $VO_SW_DIR/local/intel/Compiler/):
       #!/bin/sh
       # This script accepts one or more parameters:
       # it's just a list of "case" statements, to
       # be tailored by each site in order to perform
       # the actions relevant to each version of
       # Intel compiler installed.
       if [ -z "$1" ]; then
         echo "ERROR: script must be called with at least 1 argument"
         echo "  Example:"
         echo "    $BASH_SOURCE IntelXE-2011 intel64"
         echo "  or:"
         echo "    $BASH_SOURCE 11.1"
         return 1
       fi
       case "$1" in
         11.1)
           echo "INFO: setup environment for $1"
           intelCompilerBasePath=$VO_SW_DIR/local/intel/Compiler
           intelCompilerVersion=11.1
           intelCompilerMinor=073
           export MANPATH=${MANPATH}:${intelCompilerBasePath}/${intelCompilerVersion}/${intelCompilerMinor}/man/en_US
           export PATH=${PATH}:${intelCompilerBasePath}/${intelCompilerVersion}/${intelCompilerMinor}/bin/intel64
           export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${intelCompilerBasePath}/${intelCompilerVersion}/${intelCompilerMinor}/lib/intel64
           ;;
         IntelXE-2011)
           echo "INFO: setup environment for $1"
           if [ -z "$2" ]; then
               echo "ERROR: setup for $1 requires an extra argument: ia32|intel64"
               return 5
           fi
           if [ ! "$2" = "ia32" -a ! "$2" = "intel64" ]; then
               echo "ERROR: setup for $1 requires either argument: ia32|intel64"
               return 10
           fi
           source $VO_SW_DIR/local/intel/Compiler/XE2011/bin/compilervars.sh $2
           ;;
         *)
           echo "ERROR: invalid Intel version, exiting"
           echo "    Installed versions at present: 11.1 IntelXE-2011"
           return 2
       esac
  • *for EUMED users:* create a directory $VO_EUMED_SW_DIR/intel/, and a script called setup.sh which accepts up to 3 arguments and just runs the above script, like the following (replace $VO_SW_DIR as appropriate for your site):
       #!/bin/sh
       # This script accepts one or more parameters:
       # it's just used to call the main setup script
       # for Intel Compilers.
       if [ -z "$1" ]; then
         echo "ERROR: script must be called with at least 1 argument"
         echo "  Example:"
         echo "    $BASH_SOURCE IntelXE-2011 intel64"
         echo "  or:"
         echo "    $BASH_SOURCE 11.1"
         return 1
       fi
       source $VO_SW_DIR/local/intel/Compiler/setupIntelCompiler.sh $1 $2 $3 
  • To verify the installation, login to a different node (like a WN), execute $VO_EUMED_SW_DIR/intel/setup.sh IntelXE -2011" and see whether you can run =icc -v or icpc -v or ifort -v
  • Enjoy!

Running installation

Execution The installation procedure consists of 6 steps

As soon as you execute the installation

  cd <path_where_you_unpacked_the_archive>
  ./install.sh --SHARED_INSTALL
the following menu will appear:

Step no: 1 of 6 | Welcome
--------------------------------------------------------------------------------
Welcome to the Intel(R) Parallel Studio XE 2011 for Linux* installation.
--------------------------------------------------------------------------------
You will complete the steps below during this installation:
Step 1 : Welcome
Step 2 : License
Step 3 : Activation
Step 4 : Options
Step 5 : Installation
Step 6 : Complete
--------------------------------------------------------------------------------
Press "Enter" key to continue or "q" to quit:

After pressing <Enter> key, the installation procedure will check for pre-requisites. The installation procedure will ask you first which kind of installation you want to perform, basically if the you want to install as root or as standard user. At the end You may obtain a screen like the one below or directly the 2nd installation step.

Step no: 1 of 6 | Options > Missing Optional Pre-requisite(s)
--------------------------------------------------------------------------------
There is one or more optional unresolved issues. It is highly recommended to fix
it all before you continue the installation. You can fix it without exiting from
the installation and re-check. Or you can quit from the installation, fix it and
run the installation again.
--------------------------------------------------------------------------------
Pre-requisite(s)
-- Intel(R) Composer XE 2011 for Linux*: unsupported OS
-- Intel(R) Inspector XE 2011: unsupported OS
-- Intel(R) VTune(TM) Amplifier XE 2011: unsupported OS
--------------------------------------------------------------------------------
1. Skip missing optional pre-requisites [default]
2. Show the detailed info about issue(s)
3. Re-check the pre-requisites

h. Help
b. Back to the previous menu
q. Quit
--------------------------------------------------------------------------------
Please type a selection or press "Enter" to accept default choice [1]:

Just press enter and continue with the following step "2"

Step no: 2 of 6 | License
--------------------------------------------------------------------------------
To continue with the installation of this product you are required to accept the
terms and conditions of the End User License Agreement (EULA). The EULA is
displayed using the "more" utility. Press the spacebar to advance to the next
page or enter "q" to skip to the end. After reading the EULA, you must enter
"accept" to continue the installation or "decline" to return to the previous
menu.
--------------------------------------------------------------------------------
IMPORTANT - READ BEFORE COPYING, INSTALLING OR USING.
Do not copy, install, or use the Materials provided under this license agreement ("Agreement"), until you
have carefully read the following terms and conditions.
...
--------------------------------------------------------------------------------
Do you agree to be bound by the terms and conditions of this license agreement?
Type "accept" to continue or "decline" to back to the previous menu:

Type accept and continue to the following step 3

Step no: 3 of 6 | Activation
--------------------------------------------------------------------------------
If you have purchased this product and have the serial number and a connection
to the internet you can choose to activate the product at this time. Activation
is a secure and anonymous one-time process that verifies your software licensing
rights to use the product. Alternatively, you can choose to evaluate the
product or defer activation by choosing the evaluate option. Evaluation software
will time out in 30 days. Also you can use license file, license manager, or the
system you are installing on does not have internet access activation options.
--------------------------------------------------------------------------------
1. I want to activate my product using a serial number [default]
2. I want to evaluate my product or activate later
3. I want to activate either remotely, or by using a license file, or by using a
license manager

h. Help
b. Back to the previous menu
q. Quit
--------------------------------------------------------------------------------
Please type a selection or press "Enter" to accept default choice [1]:

Take your activation number from the email sent by the Intel web site and paste it into the next prompt; then press <Enter> key

Please type your serial number (the format is XXXX-XXXXXXXX):

If the activation key will be accepted you should see the following output:

Activation completed successfully.

Then just press <Enter> key to proceed with the next step 4

Step no: 4 of 6 | Options
--------------------------------------------------------------------------------
You are now ready to begin installation. You can use all default installation
settings by simply choosing the "Start installation Now" option or you can
customize these settings by selecting any of the change options given below
first. You can view a summary of the settings by selecting
"Show pre-install summary".
--------------------------------------------------------------------------------
1. Start installation Now
2. Change install directory [ *Your intel base path* ]
3. Change components to install [ All ]
4. Show pre-install summary

h. Help
b. Back to the previous menu
q. Quit
--------------------------------------------------------------------------------
Please type a selection or press "Enter" to accept default choice [1]:

The user can modify from this menu the installation path and the software components. As already stated in the previous section, you'd better choose a shared path here, so any VO may benefit from this installation.

*IMPORTANT* You can install wherever you see fit, but please remember to create the script $VO_EUMED_SW_DIR/intel/setup.sh when the installation is finished, ensuring that eumed users may access it for reading and execution.

Selecting the option: 1 you will go to the installation step: 5

Step no: 5 of 6 | Installation
--------------------------------------------------------------------------------
Each component will be installed individually. If you cancel the installation,
components that have been completely installed will remain on your system. This
installation may take several minutes, depending on your system and the options
you selected.
--------------------------------------------------------------------------------
Installing Intel(R) C++ Compiler XE version 12.0 for Linux*
(for applications running on IA-32) component... done
--------------------------------------------------------------------------------
Installing Intel(R) C++ Compiler XE version 12.0 for Linux*
(for applications running on Intel(R) 64) component... done
--------------------------------------------------------------------------------
Installing Intel(R) Fortran Compiler XE version 12.0 for Linux*
(for applications running on IA-32) component... done
--------------------------------------------------------------------------------
Installing Intel(R) Fortran Compiler XE version 12.0 for Linux*
(for applications running on Intel(R) 64) component... done
--------------------------------------------------------------------------------
Installing Intel(R) Debugger version 12.0 for Linux*
(for applications running on IA-32) component... done
--------------------------------------------------------------------------------
Installing Intel(R) Debugger version 12.0 for Linux*
(for applications running on Intel(R) 64) component... done
--------------------------------------------------------------------------------
Installing Intel(R) Math Kernel Library 10.3 for Linux*
(for applications running on IA-32) component... done
--------------------------------------------------------------------------------
Installing Intel(R) Math Kernel Library 10.3 for Linux*
(for applications running on Intel(R) 64) component... done
--------------------------------------------------------------------------------
Installing Intel(R) Integrated Performance Primitives 7.0 for Linux*
(for applications running on IA-32) component... done
--------------------------------------------------------------------------------
Installing Intel(R) Integrated Performance Primitives 7.0 for Linux*
(for applications running on Intel(R) 64) component... done
--------------------------------------------------------------------------------
Installing Intel(R) Threading Building Blocks 3.0 core files and examples for
Linux* component... done
--------------------------------------------------------------------------------
Installing Inspector XE Command line interface component... done
--------------------------------------------------------------------------------
Installing Inspector XE Graphical user interface component... done
--------------------------------------------------------------------------------
Installing Amplifier XE Command line interface component... done
--------------------------------------------------------------------------------
Installing Amplifier XE driver kit component... done
--------------------------------------------------------------------------------
Installing Amplifier XE user interface component... done
--------------------------------------------------------------------------------

If everithing goes ok, the installation procedure will prompt the phase 6

Step no: 6 of 6 | Complete
--------------------------------------------------------------------------------
Thank you for installing and using the
Intel(R) Parallel Studio XE 2011 for Linux*

Reminder: Intel(R) VTune(TM) Amplifier XE users must be members of the "vtune" 
permissions group in order to use Event-based Sampling.

Support services start when you install or activate your product, so please
create your Subscription Service support account now to take full advantage of
your product purchase. This support account provides access to free product
updates, interactive issue management, technical support, sample code, and
documentation.

To create your support account, visit
[[https://registrationcenter.intel.com/RegCenter/registerexpress.aspx?clientsn=]] *<your activation code>*

To get started using Intel(R) VTune(TM) Amplifier XE 2011:
   - Add the product bin64 (or bin32) directory (located in
     <your installation path>/vtune_amplifier_xe_2011) to
     your PATH environment variable.
     To start the graphical user interface: amplxe-gui
     To use the command-line interface: amplxe-cl
   - To view a table of getting started documents:
     <your installation path>/vtune_amplifier_xe_2011/
     documentation/en/documentation_amplifier.htm.
To get started using Intel(R) Inspector XE 2011:
   - Add the product bin64 (or bin32) directory (located in
     <your installation path>/inspector_xe_2011) to
     your PATH environment variable.
     To start the graphical user interface: inspxe-gui
     To use the command-line interface: inspxe-cl
   - To view a table of getting started documents:
     <your installation path>/inspector_xe_2011/
     documentation/en/documentation_inspector_xe.htm.
To get started using Intel(R) Composer XE 2011:
   - Set the environment variables for a terminal window using one of the
     following (replace "intel64" with "ia32" if you are using a 32-bit
     platform).
     For csh/tcsh:
        $ source<your installation path>/bin/compilervars.csh intel64
     For bash:
        $ source <your installation path>/bin/compilervars.sh intel64
     To invoke the installed compilers:
        For C++: icpc
        For C: icc
        For Fortran: ifort
     To get help, append the -help option or precede with the man command.
   - To view a table of getting started documents:
     <your installation path>/composerxe-2011/Documentation/en_US/documentation_c.htm.

To view movies and additional training, visit
http://www.intel.com/software/products.

Publishing the software tag

After the installation remember to publish (ask an SGM) the software tag in grid using the command

lcg-tags --ce <yourCE> --vo eumed --add --tags VO-eumed-IntelXE-2011 

Using the Intel Compiler software

FOR EUMED USERS dont forget to prepare the setup script as explained in the "Before You Install" section above. Users willing to use the Intel Compiler are expected to put, at the very beginning of their scripts, the following line:

source $VO_EUMED_SW_DIR/intel/setup.sh IntelXE-2011 intel64
to ensure all environment variables are properly set.

TESTING The Applications

Once the applications are installed it is better to test the them.

To test the application in a site
download the last version of the tar file TEST_JOBS_V<>

untar the file using
>tar xvfz TEST_JOBS_V20110608.tar.gz
>cd TEST_EUMED

The directory TEST_EUMED contains a directory for each application included in the package
To test the application go to proper directory and submit the jdl

(It is better to submit the jdl using a standard proxy and not the SGM proxy)

For example to test mrbayes in the CE ce-02.roma3.infn.it

>cd mrbayes
>glite-wms-job-submit -o ID_jobs -a -r ce-02.roma3.infn.it:8443/cream-pbs-eumed testmrbayes.jdl
Then wait for the job execution ,retrieve the output and look for 'Test done successfully!'

Topic attachments
I Attachment Action Size Date Who Comment
elsegz TEST_JOBS_V20110608.tar.gz manage 2845.1 K 2011-06-08 - 10:10 FedericoBitelli  
elsegz grid-swinstall_20130516.tar.gz manage 18.0 K 2013-05-16 - 15:56 FulvioGaleazzi grid-swinstall package, for simplified software packages installation in GRID: version 2013-05-16
Topic revision: r52 - 2013-05-16 - 15:56:50 - FulvioGaleazzi
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback