Quick Installation Guide of Utsusemi for Ubuntu 20.04/22.04LTS#

Author

Yasuhiro Inamura

Last update

Nov. 20th, 2022

Utsusemi version

4.0 or later

In this document, we describe the installation of Utsusemi using with debian packages and settings to use Utsusemi.

Supported Ubuntu#

The following Ubuntu linux versions are supported

  • 20.04 LTS on amd64 (x86_64)

  • 22.04 LTS on amd64 (x86_64)

Supported Beam Lines at MLF#

Beam lines supported by Utsusemi debian pakages are below.

BL(code)

Notes

status

BL01(SIK)

None

Ready

BL02(DNA)

not including QENSfit

Ready

BL14(AMR)

None

Ready

BL11(HPN)

not including Teuse

Ready

BL15(SAS)

not including Dview

Ready

Installation#

Utsusemi Installer for Ubuntu is distributed as a debian pakage (.deb) file. Users have only to download the packages and to execute installation command as below.

Note

Before installation, you must confirm the target PC is connected to the internet.

Installation proceedure is these,

  • Delete previous version of Utusemi if you did not install it using debian packages.

  • Update Ubuntu repository information.

  • Installation Utsusemi

    • Quick method to install all beamlines

    • Or install each package on manual

Delete previous Utsusemi#

Utsusemi will be installed here ( as default )

/opt/mlfsoft/

If this folder does not exist or you have never installed Utusemi before, you can ignore this section.

When you installed Utsusemi by using of debian packages, you can go next section. Debian package installation overwrites the previous codes.

If you are not sure about the way to install Utsusemi, you should execute this command to confirm it.

$ sudo dpkg -l mlf-manyo1

If Utsusemi was installed by Debian packages, this command returns message like below.

$ sudo dpkg -l mlf-manyo1
[sudo] password for hoge:
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name             Version            Architecture Description
+++-================-==================-============-==========================>
ii  mlf-manyo1:amd64 0.3.4.210713-ubu20 amd64        Manyo Library

When not installed by Debian packages, the command shows below.

$ sudo dpkg -l mlf-manyo1
[sudo] password for hoge:
dpkg-query: no packages found matching mlf-manyo1

If Utsusemi was installed by installation script (not debian package) as above case, you have to delete installed files as below.

$ sudo rm -rf /opt/mlfsoft

Update Ubuntu#

Before installation, you must pdate Ubuntu packages and repository information.

Execute this command.

$ sudo apt update

Quick Installation#

We prepare a script to install latest version of Utsusemi for quick installation. This script downloads all debian packages of latest version to install, instead of each package installation.

Execute these commands on Ubuntu 20.04/22.04 with Intel / AMD CPU (x86_64).

$ wget https://cdn.mlf.plus/ce/Utsusemi4/Downloads/installUtsusemi_FromDebianPackages.sh
$ sudo bash ./installUtsusemi_FromDebianPackages.sh
[sudo] password for xxx:

where installUtsusemi_FromDebianPackages.sh is Utsusemi installation script.

By running this script, all Utsusemi debian packages suitable to Ubuntu version are downloaded at current folder and installed them automaticaly.

During installation, many Ubuntu packages (required from Utsusemi) are installed and it takes a lot of time to finish all.

After installation, you can delete files used ( .deb and installation script) .

Utsusemi debian packages are below.

Packages :widths: 5 25 35 25 10 :align: center#

Package Name

Installed software

required package

BL

1

mlf-manyo1

Manyo Library

All BLs

2

mlf-utsusemi-with-ugao

Whole software of Utsusemi, including uGao (Visualizations)

mlf-manyo1

All BLs

3

mlf-utsusemi-sas

BL15 environment on Utsusemi.

mlf-utsusemi-with-ugao

BL15

4

mlf-utsusemi-dna

BL02 environment on Utsusemi.

mlf-utsusemi-with-ugao

BL02

Manual Installation#

When you do not want to install unnecessary packages, you can install each package on manual.

Download packages#

Pacakge files can be downloaded from Utsusemi installation portal ( LINK ).

Required packages for each beamline#

Beamline

Packages

BL01, BL11, BL14

mlf-manyo1, mlf-utsusemi-with-ugao

BL02

mlf-manyo1, mlf-utsusemi-with-ugao, mlf-utsusemi-dna

BL15

mlf-manyo1, mlf-utsusemi-with-ugao, mlf-utsusemi-sas

Actual package file names to be downloaded are shown below (just example).

Example packages file name#

Package File Name (for Ubuntu 20.04)

1

mlf-manyo1_0.3.4.210713-ubu20_amd64.deb

2

mlf-utsusemi-with-ugao_4.0.210713-ubu20_amd64.deb

3

mlf-utsusemi-sas_4.0.210713-ubu20_amd64.deb

4

mlf-utsusemi-dna_4.0.210713-ubu20_amd64.deb

where 0.3.4 and 4.0 means the release-version, which is given by a large update, and 210713 is the latest update day. ubu20 means this pakcages are for Ubuntu20.04. amd64 is an architecture of CPU.

Installation of packages#

To install Manyo Library, execute command using the downloaded mlf-manyo1 package ( example : mlf-manyo1_0.3.4.210713-ubu20_amd64.deb ) .

$ sudo apt install ./mlf-manyo1_0.3.4.210713-ubu20_amd64.deb

You must give the file path to deb package file to be installed. In above, ./ is used as a path to this package file because this command is executed in same folder as where package file is placed. Therefore, you will meet the Unable to locate package if you execute a command as below.

$ sudo apt install ./mlf-manyo1_0.3.4.210713-ubu20_amd64.deb

This command shows logs and the confirmation to install pacakges required by Manyo library.

$ sudo apt install ./mlf-manyo1_0.3.4.210713-ubu20_amd64.deb
[sudo] password for hoge:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'mlf-manyo1' instead of '/media/psf/Home/Ubuntu20.04/mlf-manyo1_0.3.4.210713-ubu20_amd64.deb'
The following additional packages will be installed:
  libaec0 libboost-program-options1.71.0 libboost-serialization1.71.0 libgfortran5 libgsl23 libgslcblas0 libhdf4-0 libhdf5-103
  libnexus1 libsz2
Suggested packages:
  gsl-ref-psdoc | gsl-doc-pdf | gsl-doc-info | gsl-ref-html libhdf4-doc libhdf4-dev hdf4-tools
The following NEW packages will be installed:
  libaec0 libboost-program-options1.71.0 libboost-serialization1.71.0 libgfortran5 libgsl23 libgslcblas0 libhdf4-0 libhdf5-103
  libnexus1 libsz2 mlf-manyo1
0 upgraded, 11 newly installed, 0 to remove and 0 not upgraded.
Need to get 4,050 kB/5,722 kB of archives.
After this operation, 27.1 MB of additional disk space will be used.
Do you want to continue? [Y/n]

Answer “Y” to continue.

Next is the Utsusemi installation by using downloaded mlf-utsusemi-with-ugao pacakge ( mlf-utsusemi-with-ugao_4.0.210713-ubu20_amd64.deb ).

$ sudo apt install ./mlf-utsusemi-with-ugao_4.0.210713-ubu20_amd64.deb

As same as Manyo library installation, many logs and the confirmation to install pacakges required by Utsusemi are shown like below.

$ sudo apt install ./mlf-utsusemi-with-ugao_4.0.210713-ubu20_amd64.deb
Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'mlf-utsusemi-with-ugao' instead of '/media/psf/Home/Ubuntu20.04/mlf-utsusemi-with-ugao_4.0.210713-ubu20_amd64.deb'
The following additional packages will be installed:
  blt fonts-lyx javascript-common libblas3 libdouble-conversion3 libjs-jquery libjs-jquery-ui liblapack3

...

  python3-pyside2.qtgui python3-pyside2.qtwidgets python3-scipy python3-tk qt5-gtk-platformtheme qttranslations5-l10n
  tk8.6-blt2.5 ttf-bitstream-vera
0 upgraded, 40 newly installed, 0 to remove and 0 not upgraded.
Need to get 43.2 MB/63.7 MB of archives.
After this operation, 243 MB of additional disk space will be used.
Do you want to continue? [Y/n]

Answer “Y” to continue.

Manyo Library and Utsusemi software are installed in the folder of /opt/mlfsoft/ .

BL15(SAS) Installation#

BL15 environment requires the other package (mlf-utsusemi-sas).

$ sudo apt install -y ./mlf-utsusemi-sas_4.0.210713-ubu20_amd64.deb

BL02(DNA) Installation#

BL02 environment requires the other package (mlf-utsusemi-dna).

$ sudo apt install -y ./mlf-utsusemi-dna_4.0.210713-ubu20_amd64.deb

In addition, you need to copy some files used for data reduction on BL02 environment into a folder under home folder (~/ana/DNA) as below.

$ mkdir -p ~/ana/DNA
$ cp -r /opt/mlfsoft/python-utsusemi/utsusemi/DNA/environ-home/ana/* ~/ana/DNA

Set up#

Creation of necessary folders#

The following folder must be prepared before use of Utsusemi, which folder stores raw data files.

Folder (Environment valuable)

Description

Data folder (UTSUSEMI_DATA_DIR)

Folder to store raw data which includs folder for each BL like SIK/, AMR/, HPN/ and so on )

The default path of this data folder is /data . Of course you can choose other directory as you like, for example in HOME folder.

You have to create folder and sub-folder in it. sub-folder is named by instrument-codes and is used to store each BL’s raw data folders.

$ sudo mkdir -p /data/SIK
$ sudo mkdir -p /data/AMR
$ sudo mkdir -p /data/HPN
$ sudo mkdir -p /data/SAS
$ sudo mkdir -p /data/DNA

If you do not use default /data folders, you have to set the environment variable UTSUSEMI_DATA_DIR .

Rule for raw data location

Raw data should be stored in the folder specified by UTSUSEMI_DATA_DIR according to the following rule.

  1. Create a folder who has the same name as the instrument code.

    For example, “SIK” for 4SEASONS (BL01) and “AMR” for AMATERAS (BL14).

  2. Place the data folder of corresponding instrument in this folder. The name of the data folder must include each Run No.

    For example, “SIK012345_20150301” for 4SEASONS.

    BL

    Code

    Example of data folder

    BL01 4SEASONS

    SIK

    /data/SIK/SIK012345_20160401

    BL02 DNA.

    DNA

    /data/DNA/DNA012345_20160401

    BL11 PLANET

    HPN

    /data/HPN/HPN012345_20160401

    BL14 AMATERAS

    AMR

    /data/AMR/AMR012345_20160401

    BL15 TAIKAN

    SAS

    /data/SAS/SAS012345_20160401

    BL19 TAKUMI

    ENG

    /data/ENG/ENG012345_20160401

Environment variables setting and requirements#

Defalt settings of the environment variables for each BL are included in template files below.

Environment Setting#

Target BL

Setting template file for each BL

BL01 (4SEASONS,SIK)

/opt/mlfsoft/python-utsusemi/utsusemi/SIK/bashrc.py3.SIK

BL10（PLANET,HPN）

/opt/mlfsoft/python-utsusemi/utsusemi/HPN/bashrc.py3.HPN

BL14 (AMATERAS,AMR)

/opt/mlfsoft/python-utsusemi/utsusemi/AMR/bashrc.py3.AMR

BL15 (SAS)

/opt/mlfsoft/python-utsusemi/utsusemi/SAS/bashrc.py3.SAS

BL02 (DNA)

/opt/mlfsoft/python-utsusemi/utsusemi/DNA/bashrc.py3.DNA

These files are a template file including default settings.

Main environment variables described in these template files for Utsusemi are listed below.

Environment Variables of Utsusemi#

Environment variables

Default value

UTSUSEMI_INST_CODE

<inst-code>

Instrument code [SIK,AMR,HPN and so on]

UTSUSEMI_DATA_DIR

“/data”

Data folder

UTSUSEMI_USR_DIR

${UTSUSEMI_USR_DIR-${HOME}}

Users folder ( HOME folder is default )

UTSUSEMI_LOG_QUIET

n

Control of log showing on data reduction.If “Y” , logs are decreased.

UTSUSEMI_MULTH

8

The number of multi processing for high performance on analysis on data reduction

UTSUSEMI_DEBUGMODE

n

Flag for debugging ( n is default )

To set thease Utsusemi environment variables, simply execute command like this.

$ source /opt/mlfsoft/python-utsusemi/utsusemi/SIK/bashrc.py3.SIK
===========================================
 Welcome to Utsusemi Environment for SIK
                             ver 4.0
===========================================
 Ana   : Sequencer
 M2Plot: M2PlotPlus
 D4mat : D4MatSlicer
 D4mat2: D4Mat2Slicer
===========================================

This setting way, however, is complicated because of long file path. Recommended is to add shell descriptions into ~/.bashrc like below.

# Utsusemi environments
# For BL01 (SIK)
if [ -f /opt/mlfsoft/python-utsusemi/utsusemi/SIK/bashrc.py3.SIK ]; then
   alias SIK="source /opt/mlfsoft/python-utsusemi/utsusemi/SIK/bashrc.py3.SIK"
fi

# For BL02 (DNA)
if [ -f /opt/mlfsoft/python-utsusemi/utsusemi/DNA/bashrc.py3.DNA ]; then
   alias DNA="source /opt/mlfsoft/python-utsusemi/utsusemi/DNA/bashrc.py3.DNA"
fi

# For BL11 (HPN)
if [ -f /opt/mlfsoft/python-utsusemi/utsusemi/HPN/bashrc.py3.HPN ]; then
   alias HPN="source /opt/mlfsoft/python-utsusemi/utsusemi/HPN/bashrc.py3.HPN"
fi

# For BL14 (AMR)
if [ -f /opt/mlfsoft/python-utsusemi/utsusemi/AMR/bashrc.py3.AMR ]; then
   alias AMR="source /opt/mlfsoft/python-utsusemi/utsusemi/AMR/bashrc.py3.AMR"
fi

# For BL15 (SAS)
if [ -f /opt/mlfsoft/python-utsusemi/utsusemi/SAS/bashrc.py3.SAS ]; then
   alias SAS="source /opt/mlfsoft/python-utsusemi/utsusemi/SAS/bashrc.py3.SAS"
fi

After adding, execute below command or restart the terminal app to activate settings.

$ source ~/.bashrc

To confirm settings, type a instrument code on the terminal to choose environment for each instrument.

Available beamline#

instrument

code

BL01 (4SEASONS)

SIK

BL02 (DNA)

DNA

BL11 (PLANET)

HPN

BL14 (AMATERAS)

AMR

BL15 (SAS)

SAS

For example, when you choose 4SEASONS environment, type “SIK”.

$ SIK
===========================================
 Welcome to Utsusemi Environment for SIK
                             ver 4.0
===========================================
 Ana   : Sequencer
 M2Plot: M2PlotPlus
 D4mat : D4MatSlicer
 D4mat2: D4Mat2Slicer
===========================================
$

When you want to set other values to environment variable, you should copy a template file ( bashrc.py3.XXX ) to rewrite the variables as your situations.

Especially, if you want to change the data folder path, you must change UTSUSEMI_DATA_DIR setting in copied setting file and rewrite ~/.bashrc to use new setting file ( but not touch it in detail in this document ).

Uninstall#

To uninstall Utsusemi, execute below commands. First is mlf-utsusemi-with-ugao, next mlf-manyo1 which order is obeyed the dependencies between packages.

$ sudo apt remove mlf-utsusemi-dna  # if you installed.
$ sudo apt remove mlf-utsusemi-sas  # if you installed.
$ sudo apt remove mlf-utsusemi-with-ugao
$ sudo apt remove mlf-manyo1

Version number confirmation#

Utusemi version can be seen in the following ways. It is useful when you report bugs or update Utsusemi.

  1. Installer file name

    Installer file name consists of two sets of numbers.

    Example

    MLFSOFT.X.Y.yymmdd

    X.Y

    Release number

    yymmdd

    Latest date for update
  2. Command of Utsusemi Shell

    Launch Utsusemi Shell, and run the following command.

    >>> import utsusemi
>>> utsusemi.__version__
'4.0.230403'

    Or,

    import UtsusemiInfo

    Then, the following information is displayed.

    ---------------------------------------------
               Utsusemi Info
---------------------------------------------
         version :  X.Y (rev yymmdd)
  Contact person : Yasuhiro Inamura
          e-mail : xxxxx.yyyyy@j-parc.jp
---------------------------------------------

    X.Y, are release number and yymmdd is the latest date for update.

Troubles#

If you feel something wrong on Utsusemi works, for example no progress of commands steps, please send a report about your situations and a log file “/tmp/mlfsoft.log” .

What must be reported#

To report your trouble, you tell us the information as below :

  • Which software or command you are using

  • What you are going to

  • What you did

  • What happened

Logs for errors#

Full logs from the beginning of command execution to error messages with UTSUSEMI_LOG_QUIET=n are helpfull to fix it.

In addition, to get more helpful informations, please set UTSUSEMI_DEBUGMODE=y and

  1. Create folders ~/ana/xml and ~/ana/tmp

  2. Set environment variables UTSUSEMI_DEBUGMODE to “y”

  3. Execute commands to occur the error

  4. Send full logs on the terminal and ~/ana folder including some temporal files.

$ mkdir -p ${UTSUSEMI_USR_DIR}/ana/xml
$ mkdir -p ${UTSUSEMI_USR_DIR}/ana/tmp
$ export UTSUSEMI_DEBUGMODE=y
$ Ana
... execute script occuring errors.