But what is it good for?

Ping(1M) on OpenSolaris Easter Egg

It's amazing what you discover whilst working a weekend shift with a bit of time to spare. Whilst perusing through the shell scripts used as part of the Sun in-house built VPN solution (Oracle are giving it the chop in favour of Cisco SSL AnyConnect client which doesn't officially work on Solaris x86 - it does with the help of openconnect though ;-) ), I discovered a little easter egg in ping(1M).

Fire up a terminal and ping a host:

$ ping google.com             
google.com is alive
$

Nothing fancy there and exactly as I expected. However, if you're a Linux/Mac user, this is NOT what you were expecting. So lets make the Linux/Mac users happy with our new little easter egg...

$ export MACHINE_THAT_GOES_PING=1
$ ping google.com                
PING google.com: 56 data bytes
64 bytes from ww-in-f104.1e100.net (209.85.229.104): icmp_seq=0. time=23.248 ms
64 bytes from ww-in-f104.1e100.net (209.85.229.104): icmp_seq=1. time=50.359 ms
64 bytes from ww-in-f104.1e100.net (209.85.229.104): icmp_seq=2. time=25.421 ms
64 bytes from ww-in-f104.1e100.net (209.85.229.104): icmp_seq=3. time=50.336 ms
64 bytes from ww-in-f104.1e100.net (209.85.229.104): icmp_seq=4. time=11.788 ms
^C
----google.com PING Statistics----
5 packets transmitted, 5 packets received, 0% packet loss
round-trip (ms)  min/avg/max/stddev = 11.788/32.230/50.359/17.331
$

Now that looks familiar to Linux/Mac users and Solaris users will recognise this as the output you get when using the -s flag.

So, whilst not really anything amazing, this little undocumented environment variable allows you to get the output Linux/Mac users are used to without the need for setting up an alias of continually remembering to add the -s flag.

I've only tested this on OpenSolaris so your experience may vary.

Links for 20 May 2010 - 30 Jun 2010

Links of interest for 20 May 2010 - 30 Jun 2010:

Oracle's Next Gen x86 Systems

The expression "actions speak louder than words" came to mind the yesterday when I heard Oracle were releasing a whole stack of new x86 hardware. A lot of people were concerned that Oracle were going to completely ditch x86 in favour of SPARC, but it looks like Oracle will be staying in the x86 business, albeit without AMD, for at least a whilst longer.

Yesterday's announcement saw a refresh of pretty much the entire current x86 rackmount server and blade range with the the existing 2 socket x86 servers getting an 'M2' version with access to the latest Intel Xeon Processor 5600 Series CPUs (formerly known as Westmere-EP). At the same time Oracle introduced two new, high end enterprise grade, 4 and 8 socket capable servers using the Intel Xeon Processor 7500 Series CPUs (formerly known as Nehalem-EX).

For those curious, check out these new bad boys (a brief breakdown of the specs of each machine can be found on the Oracle x86 platform blog)...

Sun Fire X2270 M2 Server

X2270 M2
Product Page | Datasheet

Sun Fire X4170 M2 Server

X4170 M2
Product Page | Datasheet

Sun Fire X4270 M2 Server

X4270 M2 12 Disk X4270 M2 24 Disk
Product Page | Datasheet

Sun Fire X4470 Server

Product Page | Datasheet

Sun Fire X4800 Server

Product Page | Datasheet

Sun Blade X6270 M2 Server Module

Product Page | Datasheet

To quote Oracle...

Based on the Intel Xeon 5600 and Intel Xeon 7500 processors, these systems are optimized for Oracle Enterprise Linux, Oracle VM Server for x86, and Oracle Solaris to provide the best performance and reliability. They are also certified to run Red Hat Enterprise Linux, SUSE Linux Enterprise Server, Microsoft Windows, and VMware.

These weren't the only new releases. At the same time Oracle announced new storage and networking blade modules.

So I think they've got most major operating systems covered and I think Oracle will be kicking their sales and marketing machines into top gear in an attempt to push as many of these out the door as possible, to see if a) they can make x86 profitable, and b) to see if they shatter the profit margins they said they'd get out of Sun following acquisition.

I still find it strange referring to what is essentially Sun hardware as Oracle hardware :-)

Auditing and OpenSSH On Solaris

Many people don't run the SSH that comes with Solaris 9 and later on their Solaris hosts, instead opting for OpenSSH or one of Tectia's SSH products. Some don't like SunSSH's versioning, as it makes it hard to determine if SunSSH is vulnerable to the same issues as OpenSSH (most often it's not or the issue has already been addressed), others rely on features on OpenSSH that haven't made it into SunSSH (there aren't many) and then there are those who's corporate guidelines only allow for a third party solution - probably for uniformity across platforms. Whatever the reason, all of these people are security conscious so they may also have an auditing (aka BSM) requirement too, and this is where the problem comes to light: they soon discover that it appears that not all events are being recorded for users who connect via this third party SSH software.

Thankfully it's easy to get OpenSSH working with Solaris auditing thanks to the very generous code contributions made by Sun to the OpenSSH community, way back in 2001, that were finally included in OpenSSH 4.0 and later. However, despite these contributions, people still miss the details on getting BSM working as they expect and this is what I'll address here.

NOTE:

These instructions are based on the latest version of OpenSSH 5.5p1 on Solaris 10, however they apply to Solaris 8 and later and all versions of OpenSSH since 4.0, when BSM support was first integrated.

When a sysadmin has been told he needs to install OpenSSH and enable auditing on their Solaris hosts, the process normally goes as follows:

  1. The sysadmin rushes over to Sunfreeware.com and downloads the necessary packages.
  2. If they use Sunfreeware.com, they follow these steps to install and configure OpenSSH
  3. They disable the OS supplied SSH, fire up OpenSSH and then test a couple of logins.
  4. Once happy OpenSSH is working, they set to configuring auditing. Not knowing exactly what they want to audit right away, they start off auditing just logins by adding lo to the flags line in /etc/security/audit_control, enable auditing by running /etc/security/bsmconv and then reboot.
  5. Once the host has come back up, they login and check their audit logs...
    $ ssh root@192.168.0.5
    root@192.168.0.5's password: 
    Last login: Wed Jun 16 13:38:45 2010
    Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
    # auditreduce -c lo | praudit -s
    Warning: invalid event no 32800 in audit trail.file,1970-01-01 01:00:00.000 +01:00,
    file,2010-06-16 16:49:30.000 +01:00,
    # 
    Huh?? No login record. More often than not, most admins then pick up the phone and call Sun Oracle asking for some help only to be told "Sorry, we don't support OpenSSH. Try with Sun's SSH and get back to us if you still have a problem. Oh yes, and don't forget to read the configuration documentation".
  6. The sysadmin then disables OpenSSH, enables SunSSH, and tests again only to find it works as they expect.

The answer Sun Oracle gave is actually the clue here. As most sysadmins don't compile OpenSSH from source (who could blame them), they miss out on the opportunity to read the README.platform file supplied with OpenSSH (it's installed in /usr/local/doc/openssh/README.platform when you install the Sunfreeware SMCosh551 pkg) and this Solaris specific snippet:

Solaris
-------
If you enable BSM auditing on Solaris, you need to update audit_event(4)
for praudit(1m) to give sensible output.  The following line needs to be
added to /etc/security/audit_event:

        32800:AUE_openssh:OpenSSH login:lo

The BSM audit event range available for third party TCB applications is
32768 - 65535.  Event number 32800 has been choosen for AUE_openssh.
There is no official registry of 3rd party event numbers, so if this
number is already in use on your system, you may change it at build time
by configure'ing --with-cflags=-DAUE_openssh=32801 then rebuilding.

Scroll back up and take another look at the auditreduce output I've included above. Now you know what is causing that warning and how to resolve it. Adding the entry above to the /etc/security/audit_event and re-loading auditing (or rebooting) will clear the message and now show your OpenSSH logins:

# auditreduce -c lo | praudit -s
file,2010-06-16 16:49:19.000 +01:00,
header,94,2,AUE_openssh,,s10vb,2010-06-16 16:49:19.278 +01:00
subject,root,root,root,root,root,932,932,0 51558 192.168.0.2
text,successful login root
return,success,0
header,89,2,AUE_logout,,s10vb,2010-06-16 16:52:15.698 +01:00
subject,root,root,root,root,root,932,932,0 51558 192.168.0.2
text,sshd logout root
return,success,0
header,94,2,AUE_openssh,,s10vb,2010-06-16 16:52:19.578 +01:00
subject,root,root,root,root,root,953,953,0 51569 192.168.0.2
text,successful login root
return,success,0
file,2010-06-16 16:52:19.000 +01:00,
#

And there we have it. Auditing working with OpenSSH. In fact, truth be told, auditing was always working here. Had the admin enabled other events to audit or logged out and then in again before checking the audit records, they would have seen an audit record for those events along with the warning.

If you happen to get output similar to:

# auditreduce -c lo | praudit -s      
file,1970-01-01 01:00:00.000 +01:00,
file,2010-06-16 16:58:49.000 +01:00,
#

... it means either your SSH implementation hasn't been compiled with auditing/BSM support, you've not enabled auditing or you've not configured it correctly so it's time to hit the documentation and test using a Sun supplied login mechanism. Don't forget that processes need to be restarted in order for them to be picked up by your auditing changes, so if in doubt, reboot.

On a side note, you may want to seriously consider using the SSH implementation that comes with the OS. Not only is it supported by the same people who provide you with the OS, it also has some features OpenSSH doesn't, most notably hardware acceleration by default. See the SunSSH community page for more details on Sun's implementation.

Do One Thing Towards Habari 0.7 Today

Michael C Harris sent this out to the habari-users group earlier today, and I thought I'd post it here to try and spread the word a bit and build up a bit of momentum.

Let's give a little push towards releasing 0.7, by doing one thing today to move Habari forward.

Here's a list of things that you could do.

Do one thing towards 0.7 today

There are a few of us who are quite eager to get Habari 0.7 out the door, but there a couple of things that need to be ironed out before it can see the light of day. So jump right in and give us a hand.

Automatic Updates for Thunderbird and Firefox on Solaris and OpenSolaris

As with Linux, Firefox and Thunderbird are available for Solaris and OpenSolaris in a variety of forms: supplied with the OS, installed separately from a pkg and installed separately from a tarball (both available from Mozilla.org). Of these, the only way you can get any sort of automatic update is with the first option: via an OS update and this tends to lag behind the available releases by quite some margin. Well, that's about to change, at least for some people.

I'll get straight into the technical details and leave the background behind all of this to the very end.

This post and the accompanying code was inspired by this article on Mozilla Developer Center.

Update: Ooops, it would seem the automatic update DOES work by default for the tarball downloads, it just doesn't offer an automatic upgrade between 3.5.x and 3.6.x as I was testing. I'll leave the rest of this post as is as an "education" for those wanting to implement their own internal crude update system.

Limitations

Lets get this out of the way early, and put it in lights:

This method only works for tarball installs in which the user running Firefox/Thunderbird and performing the update has write permissions to the directory containing the Firefox/Thunderbird installation, like their home directory.

If you don't meet this single one requirement, then the rest of this post is purely informational to you.

Requirements

The requirements are quite modest. All you need is a web server capable of running PHP (with allow_url_fopen enabled) and ideally with HTTPS/SSL support. This web server can be on an internal private network, but it needs to be able to communicate with the Mozilla.org servers.

The HTTPS/SSL requirement is not essential, but highly recommended if your web server will be accessed across a public network...

Every user will ping the update server regularly whether there's an update or not (once a day by default). Any user who connects from outside your protected network - particularly from a public WiFi hotspot - can potentially have their connection hijacked and be fed a malicious update. SSL protects against this attack. The update.xml files are small, don't sweat the SSL overhead.

The large updates themselves can be safely served from a non-secure server because the update files contain a hash that the client will verify. The hash can be trusted only if the update.xml is served securely.
MDC - Setting up an update server

Implementation and Usage

These steps assume your web server is already up and running and configured for PHP.

  1. Place this file (moz-update.phps) in a directory on a webserver, rename it by removing the "s" in ".phps" and note it's URL. You can test the script using a URL similar to the following: https://localhost/moz/moz-update.php?a=Firefox&v=3.5&b=SunOS_5.11_x86-sunc Change "localhost/moz" to correctly reflect the server name and path. You'll note I've not used "SEO friendly" URLs. I've done this deliberately as it adds and entirely unnecessary configuration step.
  2. Edit the user's prefs.js and add the following line: user_pref("app.update.url.override", "https://localhost/moz/moz-update.php?a=%PRODUCT%&v=%VERSION%&b=%BUILD_TARGET%"); As above, change "localhost/moz" to correctly reflect the server name and path.
  3. Restart Firefox/Thunderbird and go to Help → Check for Updates and sit back and watch Firefox/Thunderbird do it's magic. Firefox/Thunderbird will continue to check for updates as they do on other operating systems and you'll be notified when an update becomes available.

Notes About the Script

The PHP script is really quite simple and should be easy enough to follow, modify and customise to suit your needs. It should work out the box, though you may want to make a few changes to suit your needs and provide a quicker response:

  1. Modify $mirror to point to your nearest mirror. Just copy and paste the URL from the Mozilla mirrors list. Be sure to check it actually has the necessary contrib/solaris_tarball directories too.
  2. Optionally modify $firefox and $thunderbird to point to the version specific latest-#.# directory you wish to use for your updates. By default the PHP script will use the latest, but it you want to restrict things to a particular branch or use a beta branch, you'll need to change this. For example, if you're using Thunderbird 3.1 beta, set $thunderbird to "latest-3.1"

I've released this script under the Apache License, Version 2 so you're free to make changes etc within the terms of this license. I'm also providing it as-is without any support or assistance, though I may offer some assistance if needed. On a side note, I don't think it would be too hard to modify the script to provide your own internal update server or provide updates for other operating systems like Linux.

Right, now we've got all the technical stuff out of the way, if you're curious to find out how I came about writing this, read on...


Some Background

Years ago I switched to almost exclusively using a Sunray for my day-to-day work, and I very quickly discovered the long lag between a new release of Firefox/Thunderbird being released and the OS being updated. It was at this time that I started running Firefox and Thunderbird from my home directory. Everytime there was a new release available, I'd check the well hidden contrib directory (http://releases.mozilla.org/pub/mozilla.org/$APP/releases/latest/contrib/solaris_tarball/) to see if a Solaris/OpenSolaris release was available. If there was, I'd download the tarball, install it into my home directory, test it, and then remove the old version. This is a bit of a schlep and I ended up writing a shell script that did this for me.

I noticed the other day that for some reason my script hadn't updated my Firefox instance in a while so I set to investigating why, and that's when I asked myself "Why am I re-inventing the wheel? Firefox/Thunderbird already has an update facility built in, why not use that?"

So I did some investigating and discovered the basic method the Mozilla apps use to determine if an update is available and the files it uses to perform the update. I then discovered that the Mozilla community doesn't actually support the Solaris/OpenSolaris builds, so the Mozilla.org update servers (AUS) don't return anything useful for the requests the Solaris/OpenSolaris instances make, even though the Mozilla archive files (.mar) exist. I then hypothesized that provided the applications receive an appropriate response, they'd perform the update as they do on Windows and OS X, so I created the PHP script mentioned in this post and sure enough, it worked a treat.

The script itself doesn't do very much. All it does is provide the XML response which points to the official location of the download files. This script effectively does what the Mozilla AUS servers do, but for Solaris/OpenSolaris. No other files need to be stored on your server.

Links for 22 Apr 2010 - 19 May 2010

Links of interest for 22 Apr 2010 - 19 May 2010:

Redesigning My Blogs With HTML5

Following the recent decision to finally pull finger and migrate this site across to Habari, I thought I'd look into using as much of the evolving HTML5 spec as I can, within the limits of what most browsers support. Now I'm going to admit that I've not read the whole spec (how much free time to you think I have? ) but I have glanced through the tags and seen there is a lot of new and potentially exciting stuff there. In this post I'm going to concentrate on the structural things I'm implementing.

My blogs, like millions of others, have always followed a very similar generic structure:


<html>
<head></head>
<body>
<div id="header">
	<!-- header stuff in here -->
</div>
<div id="content">
	<!-- content stuff in here -->
	<div class="post"><!-- blog post --></div>
	<div class="post"><!-- blog post --></div>
	<div class="post"><!-- blog post --></div>
</div>
<div id="sidebar">
	<!-- sidebar stuff in here -->
</div>
<div id="footer">
	<!-- footer stuff in here -->
</div>
</body>
</html>

Now there is nothing wrong with this and it suits my needs. You can clearly see the intention of the different sections. However from a functional markup perspective they don't really differ: they're all using the same <div> tag. This is primarily because HTML4 has no method to differentiate these functions in the markup. HTML5 does and this is what I'm going to start using.

Whilst browsing through the spec, I discovered HTML5 now has some new structural type tags that could be used to differentiate the various sections in the markup: <header>, <footer>, <article> and <nav>. Each of these can be used multiple times within a page.

The working draft of the spec conveniently includes a brief usage description for each tag:

<header> :

The header element represents a group of introductory or navigational aids. [...] The header element can also be used to wrap a section's table of contents, a search form, or any relevant logos.
- W3C Working Draft

<footer> :

The footer element represents a footer for its nearest ancestor sectioning content or sectioning root element. A footer typically contains information about its section such as who wrote it, links to related documents, copyright data, and the like.
- W3C Working Draft

<article> :

The article element represents a component of a page that consists of a self-contained composition in a document, page, application, or site and that is intended to be independently distributable or reusable, e.g. in syndication. This could be a forum post, a magazine or newspaper article, a blog entry, a user-submitted comment, an interactive widget or gadget, or any other independent item of content.
- W3C Working Draft

<nav> :

The nav element represents a section of a page that links to other pages or to parts within the page: a section with navigation links.
- W3C Working Draft

The little snippets I've included above don't do each of the tags justice, so if you want to find out more about their usage, check out the relevant entries in the spec.

Now with the knowledge of these tags in mind, I'm implementing the following basic structure:


<html>
<head></head>
<body>
<header>
	<!-- header stuff in here -->
</header>
<div id="content">
	<!-- content stuff in here -->
	<article><!-- blog post --></article>
	<article><!-- blog post --></article>
	<article><!-- blog post --></article>
</div>
<nav>
	<!-- sidebar stuff in here -->
</nav>
<footer>
	<!-- footer stuff in here -->
</footer>
</body>
</html>

This is just a brief outline. The <header> and <footer> usage is self explanatory, but you may be wondering why I've used <nav> for the sidebar. Well, it's because the content of the side bar is mainly navigational in nature rather than actually being content.

I've use a <div> for the content because that's what the <div> tag is for. There is now the <section> tag in HTML5, but by my understanding of it in the spec, this wouldn't be an appropriate use for it here.

There are however a couple of catches with experimenting with new evolving stands. I encountered two in particular:

  1. I encountered display and layout problems with the new <header>, <footer> and <nav> tags until I made them block elements within my stylesheet using: header, footer, nav, article {display:block;} Some research revealed this has a lot to do with the fact that most browsers don't have default styles information for these elements, so I had to add them myself.
  2. IE doesn't display elements it doesn't know about due to the way it parses the DOM (more details here). Thankfully, this is easy enough to workaround BUT it requires the user have Javascript enabled. This should hopefully be resolved when IE 9 comes out with it's HTML5 support

Thanks to the intentions of the of the <header>, <footer> and <nav> tags within HTML5, each of these are likely to be used multiple times in a single page so I'll probably need to add some identifiers when it comes to styling them.

As I go through the process of learning more about HTML5 as I redesign this site, I'll be implementing the changes on my personal site too. You won't see these changes here until I finally flick the switch, but I've already implemented the above basic structure there, so you can take a look at the source, if you're interested.

Starting Another Move To Habari

Well, it's time I started the slow Wordpress to Habari migration process again, this time with my last remaining Wordpress site: lildude.co.uk, aka this one.

I'm now a member of the PMC and have been "eating my own dog food" ie running other sites on Habari, for quite some time so I really don't have an excuse not to migrate this site across. I could have done it a long time ago, but really didn't have the motivation. For some strange reason some of that motivation has sprouted from somewhere so I'm starting the process today.

I say process as I'm literally starting from scratch today and as with the migration of my personal site, I've got some pedantic desires and ideas that I need to consider and implement as I go along. So over the next few weeks (hopefully not months) you may notice a few subtle changes around here, but hopefully I won't break anything. Of course this will also give me some topics to write about as I migrate.

The first change: bye-bye www and trailing slash.

Top