Yet Another Linux Blog

Rayson announced a new release of Open Grid Scheduler. But the announcement does not yet have binaries or source! I was impatient and checked out the latest SVN from the sourceforge page. I ended up with revision 70.

svn co https://gridscheduler.svn.sourceforge.net/svnroot/gridscheduler gridscheduler

On a stock Ubuntu 11.04 box (well, with build-essential installed) I only had to install libpam0g-dev and also download the latest hwloc release from the official hwloc page. I untarred the hwloc code into 3rdparty/hwloc and modified the aimk.site to update the version number for hwloc.

After that, following the official instructions worked just fine, though I did also add “-no-qmon” to each aimk line.


~/source$ find . -type f -name *execd
./LINUXX64/sge_execd
./dist/install_execd


This is a guide for how to compile the latest (as of 2011-11-01) Grid Engine from source on a stock CentOS 5 machine. I followed the official instructions, with only minor modifications: Official Doc

I started out with a fresh EC2 instance, using AMI ami-eb2273ae, which is a standard CentOS 5.4 AMI from RightScale.

I ran ‘yum upgrade’ which bumped me up to “CentOS release 5.7 (Final)” and then I needed ‘yum install pam-devel.x86_64 ncurses-devel.x86_64′ in order for the compilation to succeed. Those were the only packages I needed to add.

I used the additional ‘-no-qmon’ flag to aimk, which worked fine, but then I got an error from the installation script later saying that binary was missing. So I just touched a file with that name there, and that worked around the complaint of the install script.

And that’s that. I really thought it would be more difficult.

Came across with on the Internet: ps -e -orss=,args= | sort -b -k1,1n | pr -TW$COLUMNS

If you think the machine is swapping a good way to check is to look at the swap i/o in the output of vmstat 2

Here’s the Ubuntu changelog:

flashplugin-nonfree (10.2.152.27ubuntu0.10.10.1) maverick-security; urgency=low

* SECURITY UPDATE: New upstream release 10.2.152.27
- debian/config, debian/postinst: Updated sha256sums and path.
- CVE-2011-0558
- CVE-2011-0559
- CVE-2011-0560
- CVE-2011-0561
- CVE-2011-0571
- CVE-2011-0572
- CVE-2011-0573
- CVE-2011-0574
- CVE-2011-0575
- CVE-2011-0577
- CVE-2011-0578
- CVE-2011-0607
- CVE-2011-0608

— Marc Deslauriers Wed, 09 Feb 2011 08:41:35 -0500

Here’s how you install it: ’sudo aptitude install flashplugin-installer’

It automatically works in pretty much all the browsers you have installed. I’m running Chromium 11.0.x on Ubuntu 10.10 64-bit.

Here’s the page to check your Flash plugin version: http://www.adobe.com/software/flash/about/

A few HPC rules of thumb:

• 1GB/s I/O throughput per 0.5TB of aggregate cluster memory
• 1MB/s I/O throughput per batch slot
• cluster nodes use as much electricity in three years as their initial purchase cost

Misc useful command: keep your du results so you don’t have to re-run them on large filesets:
du -sm * | sort -n | tee ~/`pwd | tr '/' '_'`.du.`date +%Y-%m-%d`

I have a USB2 dock that accepts SATA drives. On Ubuntu 10.10, just writing raw data to the SATA disks results in just about 22-28MB/s streaming write. I used ‘dd if=/tmp/test_file_1G of=/dev/sdb bs=1M’. The first time I got around 22MB/s, then the source file was in the kernel’s disk cache, so on subsequent runs, the speed jumped up to ~28MB/s. Well below the almost ~70MB/s streaming write that the drive can do (according to other posts on the web).

Then I formatted the disk with ext2 and mounted it and tried copying that same source file there with rsync. The transfer speed was ~21-25MB/s over a bunch of iterations.

The newest 64-bit build of Adobe Flash Player is the “Square” beta, downloadable here: http://labs.adobe.com/downloads/flashplayer10_square.html

After untarring and moving the libflashplayer.so into my /usr/lib64/chromium-browser/plugins the Adobe test page shows I’m running Flash version 10,3,162,29. I’m using the Chromium daily builds PPA for Ubuntu: https://launchpad.net/~chromium-daily/+archive/ppa so my current Chromium version is 10.0.607.0(68819)

This is on Ubuntu 10.10 64-bit desktop.

This version crashes less often than the built-in Chrome flashplayer. But sadly the “about:flags” click-to-play setting doesn’t seem to have any effect on the 10.3 flashplayer.

I put in an Intel SSDSA2MP040G2K5 into my Lenovo X100e, and did a clean install of Ubuntu 10.10 with the default ext4 settings.

I wanted to check whether TRIM was enabled and working. Here’s the guide to check that. Turns out TRIM was not turned on by default on my install. Here’s the guide to turn it on.

Short version: add “discard” to your ext4 mount options.

Here’s the description of the mount options for ext4: http://www.mjmwired.net/kernel/Documentation/filesystems/ext4.txt

Here’s what my system has in the kernel log after enabling TRIM:

root@x100e:~# dmesg | grep EXT4
[ 1.862448] EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: (null)
[ 2.412732] EXT4-fs (sda1): re-mounted. Opts: discard,errors=remount-ro
[ 7.271710] EXT4-fs (sda1): re-mounted. Opts: discard,errors=remount-ro,commit=0
[ 9.685766] EXT4-fs (sda1): re-mounted. Opts: discard,errors=remount-ro,commit=0

“commit=0″ is the same as “commit=5″ (default)

I didn’t make any other changes to the mount options. “noatime” and “data=writeback” could provide a bit of a performance boost, but I like having the access time information for my files, and I’ve already had a few hard lockups with this machine, so I’d rather not risk any data loss or fs corruption with the writeback mode.

A Macworld article from Sep 16, 2010 notes that even though Apple has only 5% of the market share, 65% of Data Robotics customers are on Apple’s platform. The article says this is surprising, but I don’t think so. Drobo is designed to be super easy to use, while being “good enough” in every other aspect. It’s not particularly quiet, but it’s not loud. It’s got crap performance, but it’s “good enough” for most uses.

But the killer feature is the ability to simply pop in disks as you need more space. Nothing else to do. No array settings to adjust, no filesystem to resize, no files to migrate around. Just pop in the disks and wait for the green light.

Another downside is that the percentage of storage that is usable is a bit lower than one would like, particularly when you’re mixing drive sizes. But I think it’s a reasonable trade-off.

The real problem is the crap performance. The Macworld article attracted 11 comments. Five of those comments complained about the performance. Two of those comments complained about poor support from Data Robotics when the units malfunctioned. One comment was neutral, and the remaining three were basically happy about ease of use.

One really wonders whether they could up the hw specs of their technology to greatly improve performance. Their initial device came out in 2007, and surely they could get much faster processors for the same price in 2010.

Check out the user reviews on Newegg to see a similar breakdown in responses.

Links: The Macworld article: “Data Robotics wants to be the Apple of the SOHO storage world”

Here are two tricks I’ve used recently. OpenSSH is indeed the swiss army knife of utilities.

Both of these tricks require a machine that has sshd running and that is on the network you want. Luckily, I have SSH access to many machines around the world.

Went outside of the country and wanted to listen to Pandora. But Pandora doesn’t allow streaming outside the US. Want to have all of your web traffic go through a machine in the US? Just use an ssh “dynamic proxy”. I use OpenSSH in conjunction with Chromium. So simply “ssh -D local_port user@host_in_the_US”. E.g. if you use 1080 for local_port, you’ll have a SOCKS5 proxy available on localhost:1080, and all traffic to that port will go to host_in_the_US and from there to the Internet. Then I also do ‘chromium-browser –proxy-server=”socks5://locahost:1080″‘ and bam, streaming Pandora (or access to any other webapp that whitelists IPs).

Second use is a reverse SSH tunnel that allows a connection to a machine that’s otherwise not accessible from the Internet. E.g. a machine on someone’s private network that can connect out, but is firewalled off from the outside. Again, you need a host on the Internet that has sshd and that the private machine can connect out to. So: hostA is on a private network. hostB is on the Internet. hostA can connect to hostB, but hostB can’t connect to hostB (because of NAT or firewall or whatever). From hostA, do “ssh -R 10022:localhost:22 user@hostB” Then port 10022 on hostB will forward traffic to port 22 on hostA (or another host on the private network if you use something other than “localhost”). Then, you can ssh to hostB, then “ssh user@localhost -p 10022″ which will actually connect you to hostA. These instructions were adapted from here.