Kernel Building!
Okay, building the kernels did eventually become a bit tedious given the problems I was having. Initially the 3.5.5 kernel I built worked, but was shortly followed by intermittent kernel crashes; no panics, no core dumps (disabled). So I immediately looked to the 3.4.X kernel, which would appear to be more stable. At this point I had opted for the cross compiling approach, given 7 hours to wait for a build was a bit tedious. To hopefully make the process of building kernels for the NAS more portable and easier, I created a script to do the job for me. It automates the job of installing the appropriate build tools (cross compilers, utilities and libraries) as well as keeping these up-to-date by running through the install process on each invocation.
After some playing, I found a base configuration that worked well, based on my 2.6 kernel config. At this point, I snap shot it and placed it with the script. The script now snap shots according to kernel versions built out of the same directory. I have the compilation time on my i7 down to 4 minutes. That's incredible given it took 7 hours on a 200 MHz ARM CPU. I supposed the biggest difference is the amount of memory available to me on the i7 and how many processes can be spawned. I am running make with unlimited parallel processes, limited only by the system load. I have this set to a load average of around 5. So initially, there is a surge in memory usage and CPU usage, before it all settles down, with all 8 cores working at an average capacity of about 80%. But to get an image and modules out in under 5 minutes is still astonishing!
Download the build-ukernel script
Stay tuned for more...
Monday, 8 October 2012
Friday, 5 October 2012
TP-WN822N AP/Master Mode
I have been running Debian on a hacked Buffalo LS500GL NAS for at least the past four years now and this little beast is still going strong. I've since upgraded the HDD to a 1TB disk and use it for various things, such as Firewall, NIS server, NFS server, DLNA server, Web server and just about any other kind of server you can cram into the 128 MB of RAM. It only has a 200 MHz ARM CPU, so it's no good for use as a transcoder, but for general services, it serves me well.
However, having just upgrades to a fibre connection, I thought I'd take advantage of the PPPoE availability and run a wireless access point, instead of having the switch plugged in. The NAS uses some 11 watts, so is really economic. Having the switch plugged in as well, draws an extra 8 or 9 watts; which, when you consider it is not really necessary, I may as well ditch it in favour of a USB access point.
Dilemma!
There is of course a dilemma. Most wireless drivers on Linux don't support access point mode. The simple work around is to use NDIS wrapper to wrap native Windows kernel drivers, providing the Windows kernel API, whilst interfacing with the Linux kernel. However, this is not possible when you are not running a supported Windows architecture, or at least one that you can finder drivers for. In this case, there are no Windows drivers for ARM.
Solution!
The solution is simple, so it would seem. Find a device that has a Linux driver that supports AP mode. These are few and far between and figuring out which of the devices will be suitable before you buy is a tough one. Luckily I was able to find a few resources that indicated that the TP-WN822N USB adapter supports AP mode using the Atheros drivers (ath9k variant). This driver is available in the 3.X.X kernels and onwards, so first port of call is to upgrade my 2.6.X kernel.
Cross compiling means running up a computer for hours on end while it builds, so I opt for building them on the NAS. It takes a lot longer, given the memory restrictions and slow CPU, but you can log in, run up '/usr/bin/screen' and leave it running, while you go off and do something else as equally constructive for 7 hours, like sleep!
So here is my procedure:
ssh root@nasdrive
exec screen
cd /usr/src/linux-3.X.X
make menuconfig
make
Installation is slightly different given the U-Boot loader that runs out of the NAS flash. For the NAS, you need to build the zImage and modify it slightly:
make zImage
mkimage -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n 'linux' -d <( devio 'wl 0xe3a01c06,4' 'wl 0xe3811031,4'; cat arch/arm/boot/zImage ) uImage.new
mv uImage.new uImage-3.X.X
mv System.map System.map-3.X.X
mv .config config-3.X.X
Once all this has been built, it can be copied to the /boot partition and the default links updated appropriately. It's no big deal if the NAS doesn't come up on reboot. You can dismantle it, remove the HDD, relink the old kernel and start over. This doesn't happen often, but when it does, is usually the result of not having built the uImage with the correct devio header.
So, let's leave it there for now. I am going to set my kernel building and will come back and report the next stages of getting the ath9k driver loaded and getting the wlan interface up for the TP-LINK adapter. Stay tuned...
However, having just upgrades to a fibre connection, I thought I'd take advantage of the PPPoE availability and run a wireless access point, instead of having the switch plugged in. The NAS uses some 11 watts, so is really economic. Having the switch plugged in as well, draws an extra 8 or 9 watts; which, when you consider it is not really necessary, I may as well ditch it in favour of a USB access point.
Dilemma!
There is of course a dilemma. Most wireless drivers on Linux don't support access point mode. The simple work around is to use NDIS wrapper to wrap native Windows kernel drivers, providing the Windows kernel API, whilst interfacing with the Linux kernel. However, this is not possible when you are not running a supported Windows architecture, or at least one that you can finder drivers for. In this case, there are no Windows drivers for ARM.
Solution!
The solution is simple, so it would seem. Find a device that has a Linux driver that supports AP mode. These are few and far between and figuring out which of the devices will be suitable before you buy is a tough one. Luckily I was able to find a few resources that indicated that the TP-WN822N USB adapter supports AP mode using the Atheros drivers (ath9k variant). This driver is available in the 3.X.X kernels and onwards, so first port of call is to upgrade my 2.6.X kernel.
Cross compiling means running up a computer for hours on end while it builds, so I opt for building them on the NAS. It takes a lot longer, given the memory restrictions and slow CPU, but you can log in, run up '/usr/bin/screen' and leave it running, while you go off and do something else as equally constructive for 7 hours, like sleep!
So here is my procedure:
ssh root@nasdrive
exec screen
cd /usr/src/linux-3.X.X
make menuconfig
make
Installation is slightly different given the U-Boot loader that runs out of the NAS flash. For the NAS, you need to build the zImage and modify it slightly:
make zImage
mkimage -O linux -T kernel -C none -a 0x00008000 -e 0x00008000 -n 'linux' -d <( devio 'wl 0xe3a01c06,4' 'wl 0xe3811031,4'; cat arch/arm/boot/zImage ) uImage.new
mv uImage.new uImage-3.X.X
mv System.map System.map-3.X.X
mv .config config-3.X.X
Once all this has been built, it can be copied to the /boot partition and the default links updated appropriately. It's no big deal if the NAS doesn't come up on reboot. You can dismantle it, remove the HDD, relink the old kernel and start over. This doesn't happen often, but when it does, is usually the result of not having built the uImage with the correct devio header.
So, let's leave it there for now. I am going to set my kernel building and will come back and report the next stages of getting the ath9k driver loaded and getting the wlan interface up for the TP-LINK adapter. Stay tuned...
Wednesday, 13 June 2012
Debian, NIS, Headaches, Wi-Fi and AutoFS
For quite some time now, there has been a lot of hoo-har surrounding startup ordering, dependencies and chain loading of autofs, ypbind and networking. Historically, it had been solved post-installation by manually modifying insserv directives in the headers of sysinit scripts then recalculating the dependency start order. Well, that or just simply modifying the rc.d start sequence. However, with the introduction and adoption of upstart, the dependencies have been fine tuned somewhat, allowing a virtual "start-ypbind.conf" script to be provided as a blocking event, blocking the start up of things that depend on ypbind. This works exactly as everyone that uses NIS and AutoFS require it to, providing the binding takes less than five seconds. Historically, binding time-out before ypbind startup was backgrounded, was around a minute. However, since the upstart edition, this time-out has been reduced to five seconds. After which, ypbind is backgrounded and upstart startup events are emitted for autofs and other NIS dependent services. When connecting to a network via a wi-fi connection, connecting to wi-fi alone can take that long. So obviously, since AutoFS is extremely sensitive to the presence of YP binding, automount fails to start and you are presented with a system in the same age old unusable state. The simple solution would be to increase the time-out and prey it makes its way into mainstream. However, wouldn't it be nice if YP could bind in less time, perhaps without there even being a presence of any networking at all?
Well, the answer is "yes". And the answer to "how we achieve it" is actually quite simple, although it does uncover another little bug with yphelper.
Solution
So how do we bind faster and without any networking? The answer is that we need to bind locally. In order to bind locally, it means running a local NIS server, but one intended as a NIS slave. This is extremely simple to set up, using the /usr/lib/yp/ypinit utility. Providing the -s option and the fully qualified hostname of the master NIS server, the NIS server will be replicated locally. Synchronisation with the master will be maintained by ypfxrd on the master NIS server, so no need to setup any nasty cron tasks to update the NIS records periodically. There are a few configuration settings to ensure the NIS server is only made available locally, but once done, a restart of NIS and autofs will see the system switched to using the local NIS server; ypwhich will confirm this.
Modify the yp.conf file to include your original entry below a duplicate of itself, with the IP or hostname of the NIS server swapped for localhost (127.0.0.1). It should look somewhat like this:
domain some.domain.com server 127.0.0.1
domain some.domain.com server 10.0.0.1
The host on 10.0.0.1 will be the master, used only as a secondary to the local server. Effectively, this is a null op, but it serves as a reminder to the real NIS server.
Make sure the only entry that is permitted in ypserv.securenets is the link local address range. Comment anything else out to ignore all other authentication requests.
That's it! Configuring NIS in this way will have YP binding completing almost instantaneously, while the networking is able to come online and autofs is able to start.
Issues
There is a bug with ypinit, in that on some NIS master configurations, it is unable to obtain a list of maps for that server. This can be remedied by modifying the script and replacing the call to yphelper with a call to ypwhich as follows:
maps=`ypwhich -m | awk '($2 == "'$MASTER'") { printf("%s ", $1) }'`
This is a variation of the original method for obtaining maps, and one that works reliably.
Well, the answer is "yes". And the answer to "how we achieve it" is actually quite simple, although it does uncover another little bug with yphelper.
Solution
So how do we bind faster and without any networking? The answer is that we need to bind locally. In order to bind locally, it means running a local NIS server, but one intended as a NIS slave. This is extremely simple to set up, using the /usr/lib/yp/ypinit utility. Providing the -s option and the fully qualified hostname of the master NIS server, the NIS server will be replicated locally. Synchronisation with the master will be maintained by ypfxrd on the master NIS server, so no need to setup any nasty cron tasks to update the NIS records periodically. There are a few configuration settings to ensure the NIS server is only made available locally, but once done, a restart of NIS and autofs will see the system switched to using the local NIS server; ypwhich will confirm this.
Modify the yp.conf file to include your original entry below a duplicate of itself, with the IP or hostname of the NIS server swapped for localhost (127.0.0.1). It should look somewhat like this:
domain some.domain.com server 127.0.0.1
domain some.domain.com server 10.0.0.1
The host on 10.0.0.1 will be the master, used only as a secondary to the local server. Effectively, this is a null op, but it serves as a reminder to the real NIS server.
Make sure the only entry that is permitted in ypserv.securenets is the link local address range. Comment anything else out to ignore all other authentication requests.
That's it! Configuring NIS in this way will have YP binding completing almost instantaneously, while the networking is able to come online and autofs is able to start.
Issues
There is a bug with ypinit, in that on some NIS master configurations, it is unable to obtain a list of maps for that server. This can be remedied by modifying the script and replacing the call to yphelper with a call to ypwhich as follows:
maps=`ypwhich -m | awk '($2 == "'$MASTER'") { printf("%s ", $1) }'`
This is a variation of the original method for obtaining maps, and one that works reliably.
Wednesday, 30 May 2012
Tech: Leap Motion
I want one of these!! Well, providing it works as shown and there is a Linux driver for it. :-)
Wednesday, 11 April 2012
xsltproc: exslt functions, and SEGV
This has a habit of coming back to bite me when I have forgotten about it, but essentially there is an issue with performing XSLT within a block. If the XSLT within the result block is too extensive and results in a large return block or node-set, then it causes xsltproc to SEGV. There is a simple way to defend against this from ever happening, but it requires a little concious thought and consistency.
To prevent the func:result from throwing a wobbly, you must ensure you encapsulate your return value in a variable where you would otherwise return the result. Thus func:result:
<func:result>
<xsl:for-each select="myns:isInteresting(exsl:node-set($nodes)/Thing)">
<xsl:value-of select="@name"/>
</xsl:for-each>
</func:result>
Becomes:
<xsl:variable name="ret">
<xsl:for-each select="myns:isInteresting(exsl:node-set($nodes)/Thing)">
<xsl:value-of select="@name"/>
</xsl:for-each>
</xsl:variable>
<func:result select="$ret"/>
To prevent the func:result from throwing a wobbly, you must ensure you encapsulate your return value in a variable where you would otherwise return the result. Thus func:result:
<func:result>
<xsl:for-each select="myns:isInteresting(exsl:node-set($nodes)/Thing)">
<xsl:value-of select="@name"/>
</xsl:for-each>
</func:result>
Becomes:
<xsl:variable name="ret">
<xsl:for-each select="myns:isInteresting(exsl:node-set($nodes)/Thing)">
<xsl:value-of select="@name"/>
</xsl:for-each>
</xsl:variable>
<func:result select="$ret"/>
Tuesday, 22 November 2011
ECMAScript ES6 - The future of JavaScript...
...or is it just:
<script type="text/python">
import YUI from "yui.py"
</script>
<script type="text/python">
import YUI from "yui.py"
</script>
Sunday, 23 October 2011
Modifying Gnome3 Login Background
I recently started using Gnome3 and found that it is seriously lacking in configuration utilities. While this is a problem for simplicity, you can still achieve most customisations through brute force. One of the most common customisations I have seen many requests for on Google, is that of customising the GDM3 login screen background.
The easiest way to understand how the login background can be modified, is to think of the login screen as a Gnome session, but with a different interface to that of the gnome-shell. The active user for the Gnome login screen is the GDM user. On Debian this user is Debian-gdm. So in order to modify the login screen background, you in fact need to modify the Debian-gdm user's gconf settings, updating the path to the desired wallpaper.
Its a fairly straight forward task, but is made complicated by the fact that the Debian-gdm user does not have a login shell defined in /etc/passwd and cannot be used as a valid user from the GDM login screen. So in order to access and modify the Debian-gdm user's gconf settings, you must switch to this user using the su command from a root login shell.
root# su - Debian-gdm -s /bin/bash
Debian-gdm$
Now, gconf settings are modified through the dconf subsystem using DBus. So to enable access to the DBus interface, you need to launch a DBus daemon for the Debian-gdm user and import it into the environment:
Debian-gdm$ $(dbus-launch | sed 's/^/export /g')
Next you need to start the dconf-service that interfaces with DBus and provides the backend for gconf.
Debian-gdm$ ps -eao user,command | awk '("'${USER:-$LOGNAME}'" == $1 && $NF ~ /^\/usr\/lib\/d-?conf\/dconf-service/) { print $NF }'
Running the above command will print the username and command line of the running dconf-service if there is one. If the dconf-service isn't running, then start it.
Debian-gdm$ /usr/lib/d*conf/dconf-service &
Now you can access the gconf settings for Debian-gdm through the gsettings utility.
Debian-gdm$ gsettings get org.gnome.desktop.background picture-uri
Don't forget picture-uri is a URI, so be sure to prefix it with file:// or http://. For all of this wrapped up into a script you can run as root, see my Google code project:
http://code.google.com/p/set-gnome3-background
The easiest way to understand how the login background can be modified, is to think of the login screen as a Gnome session, but with a different interface to that of the gnome-shell. The active user for the Gnome login screen is the GDM user. On Debian this user is Debian-gdm. So in order to modify the login screen background, you in fact need to modify the Debian-gdm user's gconf settings, updating the path to the desired wallpaper.
Its a fairly straight forward task, but is made complicated by the fact that the Debian-gdm user does not have a login shell defined in /etc/passwd and cannot be used as a valid user from the GDM login screen. So in order to access and modify the Debian-gdm user's gconf settings, you must switch to this user using the su command from a root login shell.
root# su - Debian-gdm -s /bin/bash
Debian-gdm$
Now, gconf settings are modified through the dconf subsystem using DBus. So to enable access to the DBus interface, you need to launch a DBus daemon for the Debian-gdm user and import it into the environment:
Debian-gdm$ $(dbus-launch | sed 's/^/export /g')
Next you need to start the dconf-service that interfaces with DBus and provides the backend for gconf.
Debian-gdm$ ps -eao user,command | awk '("'${USER:-$LOGNAME}'" == $1 && $NF ~ /^\/usr\/lib\/d-?conf\/dconf-service/) { print $NF }'
Running the above command will print the username and command line of the running dconf-service if there is one. If the dconf-service isn't running, then start it.
Debian-gdm$ /usr/lib/d*conf/dconf-service &
Now you can access the gconf settings for Debian-gdm through the gsettings utility.
Debian-gdm$ gsettings get org.gnome.desktop.background picture-uri
Don't forget picture-uri is a URI, so be sure to prefix it with file:// or http://. For all of this wrapped up into a script you can run as root, see my Google code project:
http://code.google.com/p/set-gnome3-background
Subscribe to:
Posts (Atom)