But what is it good for?

HOWTO: Set Resource Controls Using Projects Instead of ulimit(1)

Many applications, like Oracle 11g, need larger than default process values for things like stack size and file descriptors and normally document it in their installation guides. Oracle 11gR2 is very vague about how to configure the "shell limits" and instead concentrates on telling you what it expects and how you can check the current values using ulimit(1).

Using ulimit(1)

Most sysadmins turn to using ulimit(1) to set the appropriate values by adding something like the following to the system-wide /etc/profile or the specific application user's profile (the comments are mine):

# Set stack size to unlimited
ulimit -s unlimited
# Set maximum file descriptors to unlimited
ulimit -n unlimited

For the curious, these are the defaults on a Solaris 10 9/10 system (using the zsh shell as it shows the ulimit(1) options):

$ ulimit -a
-t: cpu time (seconds)         unlimited
-f: file size (blocks)         unlimited
-d: data seg size (kbytes)     unlimited
-s: stack size (kbytes)        8192
-c: core file size (blocks)    0
-n: file descriptors           256
-v: virtual memory size (kb)   unlimited
$ 

This is all well and good and works. There are however a few disadvantages with this approach:

  1. It adds another location to configure settings for your application which you may forget about.
  2. If placed in the /etc/profile, these limits will affect ALL users and processes on the system unless you add in various checks in your /etc/profile to try and limit the users this applies to.
  3. This is an antiquated method of setting these settings. Projects are the way to go. You're already using projects to set the semaphore and shared memory settings (here in the Oracle 11gR2 docs), so why not use this same functionality for setting these shell limits?

Using Projects

So how do you do it in projects?

Each of these ulimit(1) options equates to a corresponding project control:

ulimit(1) option and explanationProjects Control Name
-t: cpu time (seconds)process.max-cpu-time
-f: file size (blocks)process.max-file-size
-d: data seg size (kbytes)process.max-data-size
-s: stack size (kbytes)process.max-stack-size
-c: core file size (blocks)process.max-core-size
-n: file descriptorsprocess.max-file-descriptor
-v: virtual memory size (kb)process.max-address-space

So it's just a matter of running the appropriate projmod(1) command to set the desired value.

For example to set the stack size to 32768 kb use:

# projmod -s -K "process.max-stack-size=(privileged,32MB,deny)" user.oracle

To set the file descriptors to 1024 use:

# projmod -s -K "process.max-file-descriptor=(privileged,1024,deny)" user.oracle

Or set both at the same time using:

# projmod -s -K "process.max-stack-size=(privileged,32768kb,deny);process.max-file-descriptor=(privileged,1024,deny)" user.oracle

Note: If you don't already have a user.oracle project, you'll need to use projadd(1M). See its man page for usage details. You can also use "friendly" values as I have done above.

Your changes will only take effect on new processes created by that user, except those created as children of processes that were started before you made the change. If you can't stop a process you can use newtask(1) to assign the running process a new task, or use prctl(1) to change these values on running processes. See the respective man pages for details on how to do this.

You can verify your changes have taken effect using ulimit(1), run as the oracle user in a new shell started after the project modifications:

$ ulimit -a
-t: cpu time (seconds)         unlimited
-f: file size (blocks)         unlimited
-d: data seg size (kbytes)     unlimited
-s: stack size (kbytes)        32768
-c: core file size (blocks)    0
-n: file descriptors           1024
-v: virtual memory size (kb)   unlimited
$

... or prctl(1) run as the oracle user in a new shell started after the project modifications...


$ prctl -n process.max-stack-size -t privileged  $$ 
process: 13152: zsh
NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
process.max-stack-size
        privileged      32.0MB      -   deny                                 -
$ prctl -n process.max-file-descriptor -t privileged  $$ 
process: 13152: zsh
NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
process.max-file-descriptor
        privileged      1.02K       -   deny                                 -
$

Setting Unlimited

One question that often comes up is...

How do I set one of these values to 'unlimited' via projects like I can using ulimit?

This isn't actually made very clear in the Resource Management documentation. You can't use the word "unlimited" as the resource controls have no concept of "unlimited". Instead, you just need to set the value to the maximum allowed system value.

So what is the maximum allowed system value? Well you could dig out the tunable parameters guide and trawl through it, but the easiest method is to use prctl(1) to query the "system" value for the appropriate resource control for your current shell.

For example, to see the maximum stack size, use:

$ prctl -P -t system -n process.max-stack-size $$
process: 13152: zsh
process.max-stack-size system 9223372036854775807 max deny -
$

You can then use this value in your projmod(1) command...

# projmod -s -K "process.max-stack-size=(privileged,9223372036854775807,deny)" user.oracle

... and then verify it as the oracle user...

# su - oracle -c "ulimit -s"
Oracle Corporation	SunOS 5.10	Generic Patch	January 2005
unlimited
#

You can find further details on using projects for resource management in the Resource Management section of the System Administration Guide: Oracle Solaris Containers-Resource Management and Oracle Solaris Zones guide.

Password Strength

Today's XKCD, "Password Strength"...

Password Strength

... has been a topic that has been on my mind for a while and I'm quite pleased to see it come up in a comic. Ever since I read this interesting article on The Usability of Passwords (it's an old story but it re-emerged or at least fell into my scope of reading in May this year) I'm finding myself using simple phrases more and more as they're more secure and easier to remember. I can't use phrases for every site and host I log into as far too many sites still seem to think complexity = security, but I do where I can.

I really hope this cartoon, and this research from this paper (PDF) by Philip Inglesant and M. Angela Sasse from University College London, (which concludes that we've trained our users to use passwords that computers can easily guess and humans can't possibly remember) will make sysadmins and website developers realise that dogmatic password policies around complexity are more likely to be detrimental to security than advantageous.

HOWTO: Determine If You Are in a Solaris Zone and its Type

Solaris zones are a great invention and a brilliant use of a system's resources, but there's always been one little niggle that has annoyed sysadmins and application vendors:

How do you determine if you are in a zone or not and if you are, what is its type?

There are a number of methods I've seen being employed from people loopback mounting a file from the global zone which contains this information to making assumptions based on the output of commands like ps(1) or zonename(1) but these are all "hacks" which require some foresight by the admin and doesn't always answer the "what type of zone" question.

Well, I can say there is an officially supported method that you can use... the pkgcond(1M) command.

Have a look at this:

$ pkgcond -n is_what /
can_add_driver=0
can_remove_driver=0
can_update_driver=0
is_alternative_root=0
is_boot_environment=0
is_diskless_client=0
is_global_zone=1
is_mounted_miniroot=0
is_netinstall_image=0
is_nonglobal_zone=0
is_path_writable=1
is_running_system=1
is_sparse_root_nonglobal_zone=0
is_whole_root_nonglobal_zone=0
$

I bet you can guess straight away where I ran this command :-)

You can of course query just one parameter:

$ pkgcond -n is_sparse_root_nonglobal_zone /
$ echo $?
0
$

Knowing this little titbit of information, you can easily incorporate it into you admin or application scripts in the quiet confidence that you will get the exact answer you want without having to cobble together some fancy script or other methods to determine if you are in a non-global zone or not and if you are, its type.

Habari is Now on GitHub

After a lot of talking - we've been talking about moving off Trac for a very very long time - and a lot of hair pulling and generally bodging things together to keep Trac up and running (thanks Mike and Chris), Habari has finally made the leap across to GitHub.

Great work guys, with special thanks going to Owen for a) finally taking the bull by the horns and ending all the talking about taking action and actually taking action and b) for spending a lot of his time performing the migration of the main Habari repos and currently open tickets from SVN/Trac to GitHub, here and here respectively.

It's been just over a week now since the move took place and other than a few teething problems (I've made a few mistakes getting to grasp with git, GitHub and submodules) everything seems to be going well. More thanks to the guys on #habari (IRC) for helping us git noobs with our questions problems too.

If you're interested in contributing to Habari and are familiar with git and GitHub, you should be right at home: you can now use the same clone, change and pull request method as you use for other GitHub projects. For those who aren't too familiar, you can find a procedure here (this link may move, but I'll hopefully remember to update this post when it does).

This should make contributing to Habari soooo much easier, should ease administration of the code base and tickets so more time can be spent on coding rather than holding the house of cards up and this should also give Habari more exposure.

Happy World IPv6 Day

World IPv6 Day Why don't you spend the day investigating how ready you are for IPv6 (you WILL have to embrace it sooner rather than later) and take part in the 24 hour "test flight" being offered by some of the big organisations like Google, Facebook and Yahoo!.

You can find more details on taking part and testing at http://worldipv6day.org/.

I'm eager to take part and actually start using IPv6, but sadly my ISP seems dead keen on living in the dark ages and doesn't seem to want to offer IPv6 to its customers just yet.

Fixing iTunes Hang on Startup on My Mac

For a very very very long time now, I've been experiencing a really annoying problem with iTunes on my Mac... everytime I start it, it hangs for quite some time and along with it comes the lovely spinning beachball of death. Well not anymore, today I finally discovered the cause of the problem... one bad MP3 file.

So how did I discover this? Well simple really...

  1. With iTunes NOT running, I started Activity Monitor
  2. I then ordered the processes by name by clicking the "Process Name" column
  3. I then started iTunes. As expected, it hung and I got my beachball.
  4. With the confirmed hang, I then located the iTunes process in Activity Monitor and selected it
  5. At the top of the Activity Monitor window is a blue "Inspect" button. I clicked this.
  6. In the window that opened, I selected the "Open Files and Ports" tab. This shows you all the files and ports this application has opened.
  7. I then scrolled down to the bottom and found the last file opened... "Live @ Trade.mp3"...

    iTunes Hang

  8. I let iTunes finish its hang, located the file within iTunes and deleted it completely.
  9. I then quit iTunes and started it again.

Bingo!!! iTunes opened up instantly and was completely responsive straight away. So I lost one MP3 file I don't listen to anymore... so what. At least iTunes is responding as it should. I'm a happy boy now.

HOWTO: Disconnect from a Zone's Console Without Dropping Your SSH Connection

This problem came up on the Oracle Community forums and I thought it was worth noting here too as it's a useful tip...

I've also learned by myself (and by chance) that the best way to "disconnect" from the ngz console after a init 5 is to issue a ^C before ~.
If I don't execute a ^C and simply use ~. I'm disconnected from the console and also from the ssh connection, which is certainly annoying.

The trick here is to use a tilda (~) for every hop you made on your way to the zone's console. So if you're connecting directly from the global zone to the zone's console (zlogin -C), then the disconnect will be ~. (tilda dot). If you're connecting from another host via ssh to the global zone and then only to zone's console, use 2 tildas and one dot, ie ~~. - the first ~ for the ssh hop and the second for the "zlogin -C" hop. If you're connecting as follows: host1host2host3zlogin -C use three tildas and one dot, ie ~~~.

Keep increasing the number of tildas as you wind your way through all your hosts :-)

HOWTO: Configure Ubuntu's Unity Launcher

So I've recently updated my Ubuntu installation to 11.04 (Natty) and I'm starting to come to terms with the new Unity interface. I think I might like it, though I've not used it in anger (ie for $work) just yet, so we'll see if I change my mind at a later date.

One thing I have discovered though is it's actually quite hard to change any of the Unity settings. I couldn't find any configuration or preference applications anywhere, and right clicking on certain locations doesn't bring up a menu from which you can select "Preferences" or similar.

Well, today, after a lot of digging, I discovered a way to change some of the Unity settings, specifically the Launcher's behaviour settings. The Launcher is the list of icons on the left of the screen by default.

Unity is actually a Compiz plugin from what I can see so accordingly, you need to configure it like you would other Compiz features...

  1. Install CompizConfig Settings Manager if it's not installed already (it probably isn't)...

    $ sudo apt-get install compizconfig-settings-manager

  2. Use Unity to find and launch the CompizConfig Settings Manager (super+a and search for Compiz):

    Find CompizConfig Settings Manager

  3. Select "Desktop" in the left column and "Ubuntu Unity Plugin" on the right:

    CompizConfig Settings Manager - Desktop

  4. Make your desired changes in the two tabs available. Behaviour:

    Ubuntu Unity Plugin - Behaviour

    ... and Experimental ...

    Ubuntu Unity Plugin - Experimental

As you can see, things are a little sparse and there doesn't seem to be very much you can change, though the one thing I wanted to change - the size - is there. Maybe we'll see more options appear in the future.

The rest of the Unity desktop behaviour seems to be governed by other Compiz plugins, so be sure to have a look around the other enabled plugins within CompizConfig Settings Manager.

The theme settings are still configured like they were before, so there's nothing new on that side of things.

Adding icons to the launcher is as simple as dragging and dropping. Removing them involves right clicking and unticking "Keep in Launcher" or simply drag it to the trash can.

I've not found any other ways to configure Unity yet, but I'll keep looking. If you know of any other configuration settings, please let me know.

ZSH Command Completion in Solaris 11 Express

I use zsh as my preferred shell. It's been a while since I last updated my zsh profile and thought I'd look into implementing some useful command completion to try and make my life easier. I was particularly interested in zfs(1M), zoneadm(1M), pkg(1M), svcs(1M) and svccfg(1M). Well, imagine my surprise when I discovered most of my work has already been done for me.

Check out the ZSH auto-completion for Solaris specific commands now available in Solaris 11 Express:

$ ls -l /usr/share/zsh/4.3.10/functions/Completion/Solaris/
total 70
-r--r--r-- 1 root bin  1500 2010-11-05 15:02 _coreadm
-r--r--r-- 1 root bin  1203 2010-11-05 15:02 _dhcpinfo
-r--r--r-- 1 root bin 20003 2010-11-05 15:02 _dladm
-r--r--r-- 1 root bin   596 2010-11-05 15:02 _dumpadm
-r--r--r-- 1 root bin  3148 2010-11-05 15:02 _flowadm
-r--r--r-- 1 root bin   805 2010-11-05 15:02 _gcore
-r--r--r-- 1 root bin   406 2010-11-05 15:02 _inetadm
-r--r--r-- 1 root bin  2276 2010-11-05 15:02 _netstat
-r--r--r-- 1 root bin   670 2010-11-05 15:02 _pfexec
-r--r--r-- 1 root bin  8481 2010-11-05 15:02 _pkg5
-r--r--r-- 1 root bin  1809 2010-11-05 15:02 _prstat
-r--r--r-- 1 root bin  3330 2010-11-05 15:02 _ps
-r--r--r-- 1 root bin   449 2010-11-05 15:02 _ptree
-r--r--r-- 1 root bin   228 2010-11-05 15:02 _savecore
-r--r--r-- 1 root bin  1842 2010-11-05 15:02 _snoop
-r--r--r-- 1 root bin  1503 2010-11-05 15:02 _svcadm
-r--r--r-- 1 root bin  1667 2010-11-05 15:02 _svccfg
-r--r--r-- 1 root bin   634 2010-11-05 15:02 _svcprop
-r--r--r-- 1 root bin  1029 2010-11-05 15:02 _svcs
-r--r--r-- 1 root bin  2684 2010-11-05 15:02 _svcs_fmri
-r--r--r-- 1 root bin   300 2010-11-05 15:02 _zlogin
$ 

There is also ZFS specific auto-completion, but sadly no zoneadm. I'll look to implement this myself.

This comes from Solaris 11 Express with SRU5 applied, though from the date stamp of these files, they've been there for quite some time. A quick search indicates these went into snv_134 so were in place long before Solaris 11 Express was released.

There don't appear to be equivalent files for bash, but I guess that's probably because Bart only created zsh auto-complete scripts and then raised bug 9657 asking for them to be integrated into Solaris, which they now have been. Thanks Bart. You've saved me a lot of time.

Interesting Links for 24 Feb 2011 - 08 Apr 2011

Top