Col's Tech

But what is it good for?

Updating Solaris 11 With an SRU via ISO is So Easy

Today I had the pleasure of testing updating one of my Solaris 11 virtual machine images to SRU1 using the ISO image for a colleague and I'm pleased to say it was incredibly easy.

Once downloaded, it was a simple matter of following the instructions in the README file for the download...

Quick Start Guide
The basic steps to use the repository copy for updating your local system is as follows. Note that "sruN" represents the number of this SRU (e.g. sru1):

1. Mount the ISO image as a filesystem:

$ mount -F hsfs {full_path_to}/sol-11-1111-sruN-incr-repo.iso /mnt

2. Add the publisher to the system:

$ pkg set-publisher -G '*' -g file:///mnt/repo solaris

3. Perform the upgrade:

$ pkg update

Of course, updating using the online method directly from the Oracle support repository (See doc 1021281.1 for how to get access) was just as easy.

Solaris 11: End of Support for Legacy Hardware

It hasn't taken long for people to trip over this one and I knew I'd see this one coming. People are finding when trying to boot or install Solaris 11 on relatively old hardware, they are getting output similar to the following:

Rebooting with command: boot cdrom - nowin
Boot device: /pci@1e,600000/pci@0/pci@1/pci@0/ide@1f/cdrom@0,0:f  File and args: - nowin
Error: 'SUNW,UltraSPARC-IIIi' is not supported by this release of Solaris.
Program terminated
{1} ok 

This is because Solaris 11 does NOT include support for any of the UltraSPARC I, II, IIe, III, IIIi, III+, IV and IV+ CPUs as documented:

End of Support for Legacy Hardware

Support for legacy systems that have included the UltraSPARC I, II, IIe, III, IIIi, III+, IV and IV+ processor architectures (as reported by the Solaris 'psrinfo -pv' command) has been removed. All Oracle SPARC Enterprise M-Series Servers and Oracle SPARC T-Series Servers will continue to be supported.

Note: Oracle Solaris 10 will continue to be a supported operating system for these affected platforms as per the current Oracle lifetime support policy.

Unfortunately, I'm not privy to the reasoning behind this decision, though I suspect it may have something to do with the end-of-life dates for the hardware involved.

Of course you can continue to run Solaris 10 on these machines as the above states. You can also run Solaris 11 on x86, but just not 32-bit CPUs as 32-bit kernel support has also been removed in Solaris 11. 32-bit applications will continue to run without change.

Useful Solaris 11 Posts and Documents You Should Read

Solaris 11

Well, Solaris 11 is now out there and I'm sure people are eager to get tinkering with the new functionality.

To help you, I'll be collecting links to some useful documents and blog posts I've found and add them to this post over the next week.

Official Oracle Links and Documents

Third Party Articles and Blog Posts

Most of these are from Oracle employees who have been actively involved in the development of Solaris 11 and most of the documentation linked above. This is the section I anticipate will grow over the coming week.

General

Desktop

Development tools

IPS packaging

SMF & FMA

Security

Zones

Dtrace

Networking

HOWTO: Upgrade from Solaris 10 to Solaris 11

Now that Solaris 11 is officially available, people are going to start seriously consider upgrading to it and one of the first questions is going to be: How do I upgrade my Solaris 10 or lower systems to Solaris 11?

Unfortunately, you CAN'T. There is no direct upgrade installer or other tool that will allow you to upgrade from earlier releases of Solaris to Solaris 11. This is primarily due to the vast changes in the packaging mechanism in Solaris 10.

To quote the Transitioning From Oracle Solaris 10 to Oracle Solaris 11 guide:

There are no upgrade methods or tools available to transition your Oracle Solaris 10 system to Oracle Solaris 11. You cannot use an installer to upgrade from Oracle Solaris 10 to Oracle Solaris 11. You must perform a fresh installation of the Oracle Solaris 11 software by using one of the installation options that are described in this chapter.

In order to start using Solaris 11, it needs to be a clean install. Once installed, you can then decide what you want to do with your Solaris 10 host and the applications running on it.

Thanks to the Solaris binary guarantee, provided your application is using documented interfaces, it should run without modification on Solaris 11, so you can manually transition your users, applications and data to Solaris 11 if you wish.

Alternatively, you can create a Solaris 10 non-global zone on your Solaris 11 host and install a flash archive of your original Solaris 10 host into the Solaris 10 non-global zone using the physical-to-virtual process.

There is no support for Solaris 8 and 9 branded zones on Solaris 11 and you can't next zones: ie you can't run Solaris 8 or 9 or even Solaris 10 native zones in a Solaris 10 zone on a Solaris 11 host.

For Solaris 8 and 9 zones, you will need to continue using Solaris 10.

For those eager to upgrade from Solaris 11 Express or Early Access (EA) to Solaris 11, the process is dead easy and really gives you an idea of things to come in the future when you need to upgrade, or update.

Cisco AnyConnect Client Failing on Ubuntu 11.10

Several weeks ago I took the plunge and upgraded my work laptop to run Ubuntu 11.10 and after switching to Gnome Shell, all has gone well and I've grown accustomed to the way Gnome 3 works.

One thing that hit me when I initially upgraded was the way the proxy settings are configured and used by Ubuntu 11.10 seemed a bit broken. The "system wide" settings weren't taking effect as they used to in Ubuntu 11.04. After a fair amount of hacking around and changing individual applications to use their own settings instead of the centralize setting, all was well. Until this morning.

This morning was the first time I've used Ubuntu 11.10 to connect to work via the Cisco AnyConnect VPN client. It used to work on 11.04 so I was very surprised to find it didn't work this morning. All attempts to connect failed with...

Useless Cisco error message

Note the message at the bottom of the window... "Unable to process response from...". Not very useful at all.

A quick search around Google revealed nothing. I downloaded and re-installed the client. No go. By now, I'd wasted about 30 mins and was now technically late for my shift, so I quickly installed OpenConnect and it worked a treat and worked all day. Performance was a bit poor at times, but I don't know if that was OpenConnect, work or my broadband.

This evening, once I'd finished my shift, I investigated why the AnyConnect client wasn't connecting. Rather than trawl through Google even more, I decided to trace executing the client from the command line...

$ strace /opt/cisco/vpn/bin/vpn connect [my.vpn.servername] 2>/tmp/error.log
Cisco AnyConnect VPN Client (version 2.5.0217) .

Copyright (c) 2004 - 2010 Cisco Systems, Inc.
All Rights Reserved.


  >> state: Disconnected
  >> warning: No profile is available.  Please enter host to "Connect to".
  >> notice: VPN Service is available.
  >> registered with local VPN subsystem.
  >> state: Disconnected
  >> notice: VPN Service is available.
VPN>   >> contacting host (my.vpn.servername) for login information...
  >> notice: Contacting my.vpn.servername.
  >> warning: Unable to process response from my.vpn.servername.
  >> error: Connection attempt failed.  Please try again.
  >> state: Disconnected
VPN> %         

Nothing useful there, as I expected, but what about the strace file? Well, it did have something interesting...

socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 13
connect(13, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("8.8.8.8")}, 16) = 0
poll([{fd=13, events=POLLOUT}], 1, 0)   = 1 ([{fd=13, revents=POLLOUT}])
sendto(13, "=*\1\0\0\1\0\0\0\0\0\0\thostname\2uk\6domain"..., 41, MSG_NOSIGNAL, NULL, 0) = 41
poll([{fd=13, events=POLLIN}], 1, 5000) = 1 ([{fd=13, revents=POLLIN}])
ioctl(13, FIONREAD, [100])              = 0

Huh?? It's attempting to resolve the proxy hostname (obfuscated to hostname.uk.domain by me) which means it's trying to connect to the proxy. Why is it trying to connect to my proxy? To try and connect to https://my.vpn.servername.

Sure enough, unsetting the https_proxy environment variable from the command line allowed me to connect using the CLI invocation of the client.

So what about the GUI? Well, as I tend to use my laptop at work more often than not, I decided to just create a wrapper script around /opt/cisco/vpn/bin/vpnui as follows:

$ sudo -i
# mv /opt/cisco/vpn/bin/vpnui{,.orig}
# cat > /opt/cisco/vpn/bin/vpnui
#!/bin/sh
unset https_proxy
/opt/cisco/vpn/bin/vpnui.orig
^d
# chmod +x /opt/cisco/vpn/bin/vpnui
# exit

Now when I start the VPN client via the GUI, it unsets the proxy just for that shell and we're A-for-Away and I can connect. I know I could just disable the proxy configuration and initiate the VPN connect, but once I'm connected to the VPN, I'd need to re-enable it again. Seems like a waste of time to me when I only need the proxy disabled for a moment.

I have no idea what Ubuntu has done with the proxy configuration in 11.10, but all I know it it's far from perfect and I was definitely not expecting it to be the culprit here.

Dennis Ritchie: The Passing of a Computing Giant

More sad news in the computing world: Dennis Richie passed away this weekend after a long battle against an unspecified illness.

The passing of Dennis Ritchie is not going to get the same attention the passing of Steve Jobs did, which is a shame as I believe Dennis Ritchie contributed more to computing in more people's lives than Steve Jobs ever did, whether people know it or not. I know for certain that I wouldn't be doing my current job had it not been for Dennis Ritchie.

For those who don't know or who aren't quite sure, Wikipedia has a nice biography...

Ritchie was best known as the creator of the C programming language and a key developer of the Unix operating system, and as co-author of the definitive book on C, The C Programming Language, commonly referred to as K&R (in reference to the authors Kernighan and Ritchie).

Ritchie's invention of C and his role in the development of UNIX alongside Ken Thompson has placed him as an important pioneer of modern computing. The C language is still widely used today in application and operating system development, and its influence is seen in most modern programming languages. UNIX has also been influential, establishing concepts and principles that are now precepts of computing.

Who knows is Solaris would ever have happened had Dennis not help develop UNIX.

I think some of the tributes to Dennis have also been good and most of them quite geeky but of them all, I think James Grimmelman's tweet sums it up beautifully...

Dennis Ritchie (1941-2011). His pointer has been cast to void *; his process has terminated with exit code 0.

Rest in Peace Dennis Ritchie.

HB-Cumulus 1.9

Well, it's been a week since I implemented my changes to the tagcloud.swf and I've not encountered any problems. I've also been in touch with MustLive who reported the original XSS and HTML Injection issues I mentioned last week and he believes my changes successfully mitigate the HTML Injection issue.

So, with that in mind, I'm pleased to announce HB-Cumulus 1.9 as the latest revision of HB-Cumulus and for a limited time only, it is the ONLY port of WP-Cumulus that is NOT vulnerable to the HTML Injection issue.

I'm in the process of writing up my changes and I'll feed them back to Roy so he can fix WP-Cumulus and in doing so, fix all the other ports where the authors, like Roy, couldn't be bothered to look into this issue themselves. I need to be quick this this though as my trial Flash CS5 license is about to expire :-)

Security Vulnerabilities in HB-Cumulus 1.4 and Lower

This week an announcement was made on Seclist.org about a XSS and HTML Injection vulnerability in all the plugins that use the original tagcloud.swf from WP-Cumulus (which includes HB-Cumulus).

The XSS vulnerability was fixed a long long time ago in HB-Cumulus - 19 December 2009 to be precise - so if you are running HB-Cumulus 1.4 or earlier, you MUST upgrade.

The HTML Injection issue hasn't and can't be resolved due to the way the plugin works. That said, I'm not sure this is really much of an issue.

In short, the way the tagcloud.swf works is it takes input in the form of a string (as an argument) or from a xml file. The only HTML tag that are used and accepted by the plugin is the <a> tag. This is how the tag cloud knows that to display and link to without actually having to rebuild the Flash file repeatedly.

Now the HTML injection vulnerability basically states that this "feature" allows anyone to set any link they like, which is correct and true. Now this could be used to insert links to malicious sites, but in order for that to happen the "hacker" would have to have control of the HTML page in which the tagcloud.swf is embedded and thus effectively using the tagcloud.swf as it was designed.

This can NOT be used to exploit the security of the site hosting the tagcloud.swf file.

I have whipped up a rebuild of the tagcloud.swf that doesn't stop this issue (I can't else the plugin wouldn't work and everyone would need to recompile the tagcloud.swf everytime they updated a post) but it does limit the effectiveness of the links and by only displaying the links if they link to a domain that is the same as that hosting the tagcloud.swf file.

I'll test this for a couple of days on my own sites (it's in place on this site already) and if I don't spot any problems, I'll release an updated version of HB-Cumulus with this updated file in it.

Interesting Links for 21 Apr 2011 - 16 Aug 2011

  • Thank you Joyent! - Some very useful contributions made to Illumos by Joyent. Pity Sun wasn't quite so accepting of changes in the past. There are some really useful changes here that really should be in Solaris. Who knows, maybe they'll be incorporated one day.
  • The Cloud is not a Silver Bullet – stu.mp - A great explanation as to why the recent Amazon outage affected so many sites. In short: bad planning by the sites that used Amazon's services.
  • The Usability of Passwords (by @baekdal) #tips - A good discussion on the use of easy to remember passwords being as secure as complex passwords without the unnecessary agro of trying to type the complex passwords.
  • End of Feature Notices for Oracle Solaris 11 Express - Take note of this list of EOLd features when considering upgrading to Solaris 11 Express, especially for older SPARC machines.
  • The Observation Deck » KVM on illumos - This is pretty impressive and something I think would be great in Solaris itself. I've downloaded the ISO. Time to get playing.

HOWTO: Set Resource Controls Using Projects Instead of ulimit(1)

Many applications, like Oracle 11g, need larger than default process values for things like stack size and file descriptors and normally document it in their installation guides. Oracle 11gR2 is very vague about how to configure the "shell limits" and instead concentrates on telling you what it expects and how you can check the current values using ulimit(1).

Using ulimit(1)

Most sysadmins turn to using ulimit(1) to set the appropriate values by adding something like the following to the system-wide /etc/profile or the specific application user's profile (the comments are mine):

# Set stack size to unlimited
ulimit -s unlimited
# Set maximum file descriptors to unlimited
ulimit -n unlimited

For the curious, these are the defaults on a Solaris 10 9/10 system (using the zsh shell as it shows the ulimit(1) options):

$ ulimit -a
-t: cpu time (seconds)         unlimited
-f: file size (blocks)         unlimited
-d: data seg size (kbytes)     unlimited
-s: stack size (kbytes)        8192
-c: core file size (blocks)    0
-n: file descriptors           256
-v: virtual memory size (kb)   unlimited
$ 

This is all well and good and works. There are however a few disadvantages with this approach:

  1. It adds another location to configure settings for your application which you may forget about.
  2. If placed in the /etc/profile, these limits will affect ALL users and processes on the system unless you add in various checks in your /etc/profile to try and limit the users this applies to.
  3. This is an antiquated method of setting these settings. Projects are the way to go. You're already using projects to set the semaphore and shared memory settings (here in the Oracle 11gR2 docs), so why not use this same functionality for setting these shell limits?

Using Projects

So how do you do it in projects?

Each of these ulimit(1) options equates to a corresponding project control:

ulimit(1) option and explanationProjects Control Name
-t: cpu time (seconds)process.max-cpu-time
-f: file size (blocks)process.max-file-size
-d: data seg size (kbytes)process.max-data-size
-s: stack size (kbytes)process.max-stack-size
-c: core file size (blocks)process.max-core-size
-n: file descriptorsprocess.max-file-descriptor
-v: virtual memory size (kb)process.max-address-space

So it's just a matter of running the appropriate projmod(1) command to set the desired value.

For example to set the stack size to 32768 kb use:

# projmod -s -K "process.max-stack-size=(privileged,32MB,deny)" user.oracle

To set the file descriptors to 1024 use:

# projmod -s -K "process.max-file-descriptor=(privileged,1024,deny)" user.oracle

Or set both at the same time using:

# projmod -s -K "process.max-stack-size=(privileged,32768kb,deny);process.max-file-descriptor=(privileged,1024,deny)" user.oracle

Note: If you don't already have a user.oracle project, you'll need to use projadd(1M). See its man page for usage details. You can also use "friendly" values as I have done above.

Your changes will only take effect on new processes created by that user, except those created as children of processes that were started before you made the change. If you can't stop a process you can use newtask(1) to assign the running process a new task, or use prctl(1) to change these values on running processes. See the respective man pages for details on how to do this.

You can verify your changes have taken effect using ulimit(1), run as the oracle user in a new shell started after the project modifications:

$ ulimit -a
-t: cpu time (seconds)         unlimited
-f: file size (blocks)         unlimited
-d: data seg size (kbytes)     unlimited
-s: stack size (kbytes)        32768
-c: core file size (blocks)    0
-n: file descriptors           1024
-v: virtual memory size (kb)   unlimited
$

... or prctl(1) run as the oracle user in a new shell started after the project modifications...


$ prctl -n process.max-stack-size -t privileged  $$ 
process: 13152: zsh
NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
process.max-stack-size
        privileged      32.0MB      -   deny                                 -
$ prctl -n process.max-file-descriptor -t privileged  $$ 
process: 13152: zsh
NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
process.max-file-descriptor
        privileged      1.02K       -   deny                                 -
$

Setting Unlimited

One question that often comes up is...

How do I set one of these values to 'unlimited' via projects like I can using ulimit?

This isn't actually made very clear in the Resource Management documentation. You can't use the word "unlimited" as the resource controls have no concept of "unlimited". Instead, you just need to set the value to the maximum allowed system value.

So what is the maximum allowed system value? Well you could dig out the tunable parameters guide and trawl through it, but the easiest method is to use prctl(1) to query the "system" value for the appropriate resource control for your current shell.

For example, to see the maximum stack size, use:

$ prctl -P -t system -n process.max-stack-size $$
process: 13152: zsh
process.max-stack-size system 9223372036854775807 max deny -
$

You can then use this value in your projmod(1) command...

# projmod -s -K "process.max-stack-size=(privileged,9223372036854775807,deny)" user.oracle

... and then verify it as the oracle user...

# su - oracle -c "ulimit -s"
Oracle Corporation	SunOS 5.10	Generic Patch	January 2005
unlimited
#

You can find further details on using projects for resource management in the Resource Management section of the System Administration Guide: Oracle Solaris Containers-Resource Management and Oracle Solaris Zones guide.

Top