Whoto know if a long process is done?

The Problem:
I sometimes run a very long process which can even reach up to few days.
Instead of looking and checking if the process has done, I needed a solution to get a message from the process itself.
One way to do it is by using mail like this:
echo "some message" | mail -s "subject of mail" yourmail@wherever
Unfortunately, for some reason it doesn't work for me.
if you have any solution - let me know. I used a workaround...

The Solution (Option I):
Using Thunderbird and cshell and a tip from here (look at the comments), I built a script that can be added to a command line or another script which run's the process.
needed ingredients:
- Thunderbird
- xdotools

usage: simply copy the command below, edit where needed and run:

mail_script.csh [few message body words]

mail_script.csh:

#!/bin/csh
# this will send a mail with subject:"mission accomplished"
# and command line variables (argv) as the message body
# the script require Thunderbird and xdotool installed
# Just change the "to" field in the command below and
# set a program to run it at the end of process
# by Ran Novitsky Nof 2009 @ BGU
set body = `echo $argv`
thunderbird -compose "to='yourmail@wherever',subject='Mission Accomplished',body=$body" & ; sleep 5 ; set WID=`xdotool search --title
"Compose" head -1`; xdotool windowfocus $WID;xdotool key ctrl+Return

 The Solution (Option II):
  Option I needs Thunderbird to be installed and it uses gui so somtimes there are conflicts with user activity. an alternative solution is to install ssmtp (most Linux distros has it as a package). after installing edit /etc/ssmtp/ssmtp.conf file:

most importent lines to edit:
Mailhub - you smtp server (e.g smtp.gmail.com or any other smtp server)
AuthUser - see conf file link above
AuthPass - see conf file link above
AuthMethod - see conf file link above
FromLineOverride - remove the # at the start of line

edit /etc/ssmtp/revaliases:
add a line:
root:[user]@[mailaccount]:[smtpserver]:[port]

and run the script mail_script.csh:

# this will send a mail with subject:"mission accomplished"
# and command line varibles (argv) as the message body
# the script requier ssmtp installed
# Just change the -F , SendTo and "To" fields in the command below and set a program to run it at the end of process
#
# by Ran Novitsky Nof 2010 @ BGU
set body = `echo $argv`
sudo ssmtp -F"name of sender" recipient@mailserver << END
To:recipient@mailserver
Subject:Mission accomplished
$body
END

Tag Clouds

The Problem:
How to replace the tag list (aka labels) to a tag cloud?
The Solution:
Details in here you can see the outcome on the sidebar of my blog.
Tip:
To add a heading to the tags cloud simply add another <div> before the div of widget-content:

new div: <div><h2 class="'title'">Labels Cloud</h2></div> (in red is the title I selected for the gadget, can be anything actually)

original script (only the first few lines):<b:widget id="'Label1'" locked="'false'" title="'Labels'" type="'Label'"><b:includable id="'main'"> [new div in here] <div class="'widget-content'"><div id="'LabelDisplay'/"></div><script language="'javascript'" type="'text/javascript'">

Orbits and other auxiliary data

I use Delft orbital data (ODR) which is available from here. don't forget to use the precise version and to download all .ODR files (make sure it is not .odr after downloading) and the arclist. also, download and install getorb software and link or move the executables to the ROI_PAC binaries directory.

I also use DORIS orbital data for Envisat which is available from here. but will require a user and password. the DORIS data is used when no ODR data is yet to be available for new Envisat images.

for Envisat I also use auxiliary data available from here.

in my lab, all the orbital data is in ~/ROI_PAC_3_0/Delft with directory tree as follows:

in ~/ROI_PAC_3_0/Delft I put the Envisat auxiliary data (INS and XCA files)

• doris/Envisat/YYYYMM contains orbital data for the year (YYYY) and month (MM)
• Envisat contains Envisat Delft orbit data (.ODR .res arclist files)
• ERS1 contains ERS1 Delft orbit data (.ODR .res arclist files)
• ERS2 contains ERS2 Delft orbit data (.ODR .res arclist files)

the files must be updated from time to time as more images are added to the list.

ROI PAC installation

UPDATE: Oct 9, 2011 (for version 3.0.1):

first download ROI-PAC and test data from open channel Foundation

I will assume the tar files are extracted at home directory (~), but it can be anywhere you wish.

don't forget to grand read/write access to relevant users.
make sure you have at least gcc and gfortran installed on your system.

go to ~/ROI_PAC_3_0_1/ROI_PAC/ directory.

follow steps described in ~/ROI_PAC_3_0_1/ROI_PAC/AAREADME_BUILD_ROIPAC up until line:
which uncompress

make sure you have a TEST DIR testing data to test the compiled program. this directory should be tared but not gzipped using:
gunzip roi_pac_test_dir.tar.gz
this will convert the roi_pac_test_dir.tar.gz to roi_pac_test_dir.tar
then create a directory test-runs in the ROI_PAC directory.
cd test_runs
and run the multitest.sh script:
../contrib/multitest.sh /FULLPATH/roi_pac_testdir.tar /FULLPATH/ROI_PAC_3_0_1/ROI_PAC/multibuild-xxxxxx-yyyy/installs/share/roi_pac /FULLPATH/ROI_PAC_3_0_1/ROI_PAC/multibuild-xxxxxx-yyyy/installs/BUILD1/bin  ... ... ... /FULLPATH/ROI_PAC_3_0_1/ROI_PAC/multibuild-xxxxxx-yyyy/installs/BUILDN/bin

where:
FULLPATH is the path to your home directory (or where ever the install is)
xxxxxx-yyyy is the multibuild date-time stamp
and BUILD1 to BUILDN are all the compilers sub directories in /FULLPATH/ROI_PAC_3_0_1/ROI_PAC/multibuild-xxxxxx-yyyy/installs/ e.g. defaults, gfortran, gfortran64 etc.

check for the fastest build by checking the batchlog files for each build in test-runs directory.

go to multibuild directory and into installs. get the bin directory from the desired compiler directory and copy it to the ~/ROI_PAC_3_0_1/ROI_PAC directory.

• note: I moved the bin directory to: ~/ROI_PAC_3_0/ROI_PAC/BIN/LIN

see the ROI_PAC Wiki - Installation for updated installation notes
look also at ROI_PAC Wiki for updates and patches.

now you can use ROI PAC, but keep in mined orbits and other auxiliary data should also be available for processing.

Installing Linux Fedora Core 10

Most of the info can be found here

I will only add some relevant steps to InSAR processing as I use it and steps I found useful.

installing gnome and most of the distro (including tcsh as it is my favorite shell)

- change nautilus to open always in browser
- remove SElinux policy
- remove option in keyboard options - allow mouse to be controlled by keypad
- add vino vnc option to user
- change autologin for user: vi /etc/gdm/gdm.schemas
- add user to sudo:

su --login -c 'visudo'
Now go below the line:
root    ALL=(ALL)       ALL
press a, and type the following
username ALL=(ALL)      ALL
where username is the username you use to login. Next press Escape.
Now, if you want to be prompted for your root password each time you use the sudo command go to this line:
# %wheel        ALL=(ALL)       ALL
and with your cursor pointing on # press x
If you don't want to be prompted for your root password each time you use the sudo command go to this line:
# %wheel        ALL=(ALL)       NOPASSWD: ALL
and with your cursor pointing on # press x
Next, press :wq to save and exit. (type the : as well)
* Adding your user to the wheel group
Just type:
su -c 'gpasswd -a username wheel'
* Testing sudo
To test if you have done it correctly as a simple user type:
$ sudo whoami
If everything is working properly the command should return the word 'root'.

- adding windows fonts:
  wget http://www.my-guides.net/en/images/stories/fedora10/msttcore-fonts-2.0-2.noarch.rpm
  su -c 'rpm -ivh msttcore-fonts-2.0-2.noarch.rpm'
- open terminal here in nautilus
  su -c 'yum install nautilus-open-terminal'
- Enable Num Lock on GNOME startup
  su -c 'yum install numlockx'
  su -c 'cp /etc/gdm/Init/Default /etc/gdm/Init/Default_backup'
  su -c 'gedit /etc/gdm/Init/Default'
  And add the following lines at the end of the file before the exit 0.
  if [ -x /usr/bin/numlockx ]; then
    /usr/bin/numlockx on
  fi

- add to install:
  yum extender
  yum Fastest Mirror Plugin
  GMT
  qgis
  grace
  filezilla
  ipython
  KDocker
  google-earth
  wine
  g77
  bluecurve icons
 gnash - for flash viewing

- setting /etc/X11/xorg.conf for display with no monitor attached:
make sure to use a video card which is not onboard (or perhaps this is only valid for the 965 intel mobo?)

- edit /etc/fstab for mounting external HD (LAN):
//XXX.XX.XX.XX/DATA /mountpoint cifs auto,owner,rw,username=XXX,uid=XXX,gid=XXX 0 0
NOTE! - it will not mount on startup. need to do: "sudo mount mountpoint" to mount. one can add password=[user-passwd] after username=[username] or instead username and password : credentials=[path to a file containing user and password]
the file should contain the remote system properties:
username=[username]
password=[userpassword]

- adding python modules
for gui: http://easygui.sourceforge.net/index.html
scipy
numpy
matplotlib
ipython
pylab

in a case of emergency: enabling write on a system error:
mount -t ext3 /dev/sda1 / -o remount,rw

- installing ROI PAC
- adding axillary data

Working on two machines at once

The Problem:
On my work I use two computers, one is Linux based which carry out all the heavy processing and another is windows based for viewing results, office related jobs and other Microsoft compatibility issues.

My problem was I had two mice, two keyboard and two screens. it is very confusing and tiring to switch from one to another.

The Solution:
The Linux distro comes with a wonderful component - VINO SERVER a VNC server which connects to your display 0 (the main display).
so I installed REALVNC viewer on my windows computer, connected one screen to the DVI exit and one to the D-sub (VGA) exit on the graphic card.
now, all need to do is to enter the Linux machine via VNC, drag the window to the second screen and use "Full Screen" view. This allows me to use one set of keyboard and mouse and move from one machine to another just by moving from one screen to another.

Tips:

• autologin:
  Since you can't "see" the Linux machine without connecting to it, you can tweak the bootloader to load you user automatically:
  edit /etc/gdm/custom.conf (as superuser if needed) and change:
  [daemon]
  TimedLoginEnable=true
  TimedLogin=yourusername
  TimedLoginDelay=0
• ssh first:
  if you don't want to use autologin for security or other reason, you can login via ssh and start a gnome session and then use the VNC viewer. just enable ssh daemon and vino server.
• othere VNC servers:
  you can always use different VNC server and connect to other displays then 0.
• graphic card:
  I tried to use this method with an onboard G. card and only got low resolution screen size.
  it might be my mother board (intel DQ965GF) or bad X settings but using a GeForce card I managed to use my desired resolution with no problems.

Welcom to My Blog.

My name is Ran Novitsky Nof. I currently work at the Geophysical Institute of Israel.

This blog's first aim is to serve as a holding place for my ideas, thoughts, tips and workflows during my research and sharing it with the world - if only one person would benefit from something here - I did my share.

Additional Info may be found at my personal homepages : ran.rnof.info