RootForum Community » Planet

Google hat bei der Aufzeichnung von Streetview Bildern WLAN Karten erstellt, genau wie Skyhook und Apple es tun. Diese Daten dienen der GPS-losen Navigation. Im Fall von Google sind dabei auch Nutzdaten von ungesicherten Wifi-Netzen mit aufgezeichnet worden, aber nie genutzt worden.

In England hat das Büro des Information Commissioners (Datenschutzbeauftragen) diese Daten jetzt stichprobenartig untersucht und dabei keine personenbezogenen Daten gefunden (Golem).

Ein Google Streetview-Auto muß sich schon recht nahe an einem offenen WLAN befinden, damit es Nutzdaten aus diesem Netz aufzeichnen kann. Betreiber solcher Netze sollten dennoch prüfen, was sie da tun, denn den Nachbarn spielt das Netz die Daten genauso ungesichert vor, aber 365 Tage im Jahr rund um die Uhr.

Anderswo präsentiert das Microsoft Bing Team inzwischen ihre Version von Street View (Youtube), Street Slide genannt.

Julian Hein hat eine eine Reihe von Stellen offen bei Netways. Netways sind die Leute, die neben anderen Sachen ganz viel Icinga, Nagios und OpenNMS machen. Alle Stellen sind Vollzeit und in Nürnberg (zum Teil Stellungstypisch mit Reisetätigkeit).

Netways hat eine Website und ein Blog. Bewerbungen gehen als PDF an jobs at netways.de, und Julian Hein beantwortet Fragen zu den Stellen gerne als jhein at netways.de oder am Telefon (+49 911 92885-0).

Armin Gnosa schreibt mir:

Hallo, Kris! Für Deine Jobs-Kategorie einmal ein Stellengesuch.

Ich suche ab dem 1. September (ggfs. auch früher) eine Stelle in der Technischen IT.

Seit meiner Jugend (das war 1994) beschäftige ich mich regelmäßig mit Netzwerken und Linux bzw. OSS im Allgemeinen. In letzter Zeit auch beruflich, jedoch leider nur im kleinen Rahmen. Man könnte sagen, daß es sich dabei durchaus um eine Leidenschaft handelt.

Was meine Kenntnisse und Interessen angeht, bin ich ziemlich breit aufgestellt, würde mich aber auch gerne in eine Richtung spezialisieren (sei es Storage, Datenbanken oder auch Security oder Spam-Abwehr).

Neben den üblichen Stärken wie Kommunikations- Konflikt- und Teamfähigkeit bringe ich auch ein hohes Verständnis für Professionalität und einen großen Lernhunger mit.

Alles weitere findet sich auf meinem Xing-Profil: https://www.xing.com/profile/ArminF_Gnosa.

Viele Grüße, ...

Armin lebt und arbeitet derzeit in Hamburg und würde eine Stellung dort in der Nähe bevorzugen.

Hey guys,

boys needs toys! That's a fact. I saw the Magic Trackpad announced by Apple this Tuesday and I ordered it instantly. It was send via Post. Not kidding, it arrived in a big white envelope and it took 2 days to get from the Netherlands (I live 40km East from the Netherlands border) to my home.

After opening the envelope, I found a nice little box inside; this one:

Opening it...

It is VERY important, that you believe in this text! Run a system update and download the drivers from this website http://support.apple.com/kb/DL1066 and reboot your Mac. Once you've rebooted, pair the Magic Trackpad with your Mac by opening the Trackpad options in your System Preferences. If you played around (like me) before and paired it using bluetooth, maybe you also need to remove the batteries for a moment before you pair it again.

The Magic Trackpad is a piece of art. Simple, elegant, "less is more" following - it's a typical Apple product and I already like this little guy ;)

Here's a photo of the Magic Trackpad aligned with my Wireless Keyboard, my Magic Mouse and - my other input device I often use, my cool pen from Lamy.

At least, here's a near view of the Wireless keyboard and the Magic Trackpad.

Resumé (so far)

It's about an hour ago since I got this beautiful new device, but so far I can say, ONCE it is installed (sorry, but today this reminds me more of a Windows gadget installation than an Apple installation/configuration), it's a great device. Sure, I guess we will need some time to get into using it on the desk, especially the none existing frame around this device makes it hard for me to find the "right click", but that's the only thing that I would put on the "I need to get used to it" list ;)

It's like a huge trackpad from your MacBook "mounted" on your desk ;) I like it, but I admit, to rework the photos you see above, I used my Magic Mouse laying on my desk as well. For regular stuff I think it's a great device.

As always, I've uploaded all shown photos to my flickr account in HiRes, so feel free to watch them on: http://www.flickr.com/photos/37773250@N02/sets/72157624614281876/

Ciao
Dennis

Permalink | Leave a comment  »

Kobo is an interesting website that sells ePub e-books, compatible with Adobe Digital Editions (so, DRM’d), of which I wrote already before — unfortunately a couple of interactions with the website and its features lately have been slightly upsetting. So I’d like to simply express my opinion about it…

Dear Kobo,

Looking for a website that would sell me ePub books I could read on my Sony Reader device, I have to say that yours is the one that is the most appealing; the other decent option for novels and non-fiction (non-technical) books is the British WHSmith, but their website is a bit difficult to grok and feels a bit.. old style.

It also helps the fact that the Dollar is still lower than the Euro, while the Pound is higher again, which means that in general – minus strange cases like the sixth Hitchhiker’s book “And Another Thing” that seems to cost more on your website than from other stores – the price is quite good for me. I’m also pretty happy that I can get books released in America but not yet in Europe, like it happened with Cyber War which I positively enjoyed.

Unfortunately, things started to get strange when I bought “The Salmon of Doubt”… and had been unable to read it on my Reader; it turned out that I had to work around the DRM to be able to find the cause of the problem: the internal format of all the other books I bought is XHTML, while that one uses DTBook. Somehow, both the Sony software and your website allow it to work, but it fails badly both on the Reader and on your own Android application. Which probably means, you also didn’t plan about it. I wonder if your eReader device actually reads those.

The other problem I noticed, was that beside a number of books not available from you, but available on, again, WHSmith, for which I obviously can’t say much (either you have them or you don’t, like any other book store), there is an annoying trouble in getting chapters of book series. For instance I could get Jim Butcher’s Grave Peril from you, but then I had to turn to WHSmith for Summer Knight, Death Masks and Blood Rites, since you don’t have them. Similarly yesterday I noted that while I got Robin Hobb’s “Assassin’s Apprentice”, the second volume (Royal Assassin) is also not available, but the third (Assassin’s Quest) is.

It is an annoyance – especially since I prefer getting books on your site also because they are available on the web to read with my Linux laptop, and on my Android phone, while WHSmith’s books I can only load on the Reader or read with the official applications, minus breaking the DRM again – but a minor one at that.

It goes a bit worse when I received last week a promotional email with a $2 discount, not a lot but since I’m actually quite through The Dragon Reborn (for the second try, the first one I abandoned two years ago in the hospital), I thought it would have been a good chance to find something new to read after that. I tried it a couple of nights ago with a few of the books I bookmarked, but.. for all of them it reported expired or not applicable.

Tonight, you reply me on twitter saying that only a subset of books are available and provide me a list … but once again, trying to apply the code to two books in that list, reports that it is expired or not available. For sure, it wasn’t expired, since the mail said it would expire at “29 June 2010 11:59pm EST” – and at the time it was something like 17 EDT – but at the same time, the mail has no reference to the list I was given nor either any reference to the discount applying to a subset of the books (well, it was understandable anyway).

At the end, I bought the one book, that I didn’t know already, from that list that looked interesting and relevant to my areas of interest; for the curious it was Free by Chris Anderson, without the discount it was still at a decent price. But all of this feels like quite the kerfuffle.

I think that it would be good for both you and your customers if you can actually get these things sorted out properly; as I said I’m very happy to continue being your customer, I don’t even care about promotional codes (after all, $2 is almost nothing), but it doesn’t feel right …

Oh and if you happen to be able to… could you please make a Linux application to download Adobe Digital Edition files? Thank-you!

In my previous post where I declared myself up for hiring by those who really really want Ruby 1.9 sooner than we’re currently planning to release it, I’ve said that the Ruby team doesn’t want to “Pull a Python 3”. I guess that I should explain a bit what I meant just there.

Ruby 1.9 and Python 3 are, conceptually, actually similar: while Python 3 actually make a much wider change in syntax as well as behaviour, both requires explicit, often non-trivial, porting of the software to work. Thus, they both require you to be slotted, installed side-by-side, with the older, more commonly used alternative, and so do the libraries and programs.

There is more similitude between the way the two are handled than you’d expect, mostly because the Python support for that has been partly copied out of Ruby NG stripped of a few features. These features are, for the most part, what I’d say protect us from pulling a Python 3.

As it is, installation of Python 3-powered packages is done once Python 3 is installed; and Python 3 is installed, unless explicitly masked, on every system, stable or not, because of the way Portage resolves dependencies. In my case, I don’t care about having it around, so it’s masked on all my systems (minus the tinderbox, for obvious reasons). You cannot decide whether a given package is installed for 2.6, 2.7 or 3.1, and you can only keep around safely one Python for the 2.x series as it will only install for that — which is going to be fun, because 2.7 seem to break so many things.

Ruby packages instead is coordinated through the use of the RUBY_TARGETS variable, that allows us (and you) to choose for which implementation (if supported) install a given package; you can even tweak it package-per-package via package.use! This, actually, makes the maintenance burden quite higher on our side because we have to make sure that all the dependency tree is up-to-date with a given target, on the other hand though it allows us be sure that the packages are available, and it would scream at us if they weren’t (or rather Mr Bones would).

Most importantly, we don’t need no stinkin’ script like python-updater to add or remove an implementation; since the implementations are user-chosen via an USE-expanded variable (RUBY_TARGETS as I said), what you otherwise do with python-updater (or even perl-cleaner) is done through …. emerge -avuDN world.

Even though, I’ll admit, there is one thing that at least python-updater seems to take into consideration and that for now we can’t cater: using the Ruby interpreter rather than binding a library to be usable via Ruby; as I said in the post I linked above, it’s one of the few cases that needs to be kinked out still before it can be unmasked. Again you can either wait or hire somebody to do the dirty job for you.

A note about the “stinkin’ script” notion: one of the reason why I dislike the python-updater approach is that it lists a few “manual” packages to be rebuilt. The reason for that to happen is the old Python bug that caused packages to link the Python interpreter statically. The problem has since been fixed, but the list (which is very limited compared to what the tinderbox found at the time), is still present.

It is not all. I said at the start that right now Python 3 is installed unconditionally by default on all systems; we’re going to do double- and triple-work to make sure that the same won’t happen with Ruby 1.9 until we’re ready to switch the defaults. Switching the defaults will likely take a much longer time; we’re going to make 1.9 stable first, and start stabling packages supporting that… from there on, we’d be considering removing packages that are 1.8-only.

Well, to be honest, we’re going to consider switching some packages that won’t work with 1.9 (or JRuby) and neither have use nor they are maintained upstream. For good or bad, a lot of the packages in the tree have been added by the previous team members, and they, like us, often did so when they had a personal interest in the package… those packages often times are no longer maintained and are dead in the water, but we still carry them around.

Anyway, once again, the road is still bumpy, but it’s not impossible; I’m not sure if we can get to unmasking it before end of the summer as I was hoping to, but we’re definitely on track to provide a good user experience for Gentoo users who develop in Ruby, and most of the time, we can even provide a better upstream experience.

Gentoo currently does not offer Ruby 1.9 available to users directly; there are a number of reasons for that, and can be summed up in what Alex described as “not pulling a Python 3 on our users”. Right now, there are near to no packages that need Ruby 1.9, and a lot that does not even work with it. While a minority nowadays, a few won’t even work if it’s installed together with 1.8, let alone configured as primary provider for Ruby.

Me, Alex and Hans have been working for a long time to find a solution, and since last year the definite solution seems to be Ruby NG which I originally started in May 2009 after having trouble with keeping this very blog alive on the previous vserver — which nowadays only hosts the xine bugzilla .

The road has been still uphill from there, as the three pages of posts tagged with RubyNG on this blog can document; trouble with the ideas and implementations, compatibility problems, a huge web of dependencies between packages, various fixes, all of it makes the road to Ruby 1.9 quite difficult for us packagers. At the same time, we’ve been doing our best to ensure that what the users are given with proper software, of good quality. Maybe it’s because I’m deeply involved with QA, maybe it is because I’m not writing production software daily, but I still think that we shouldn’t be providing with half-assed software easily, just for the sake of it.

That means that most of the time we either don’t add support for Ruby 1.9, or we go deeply into fixing the underlying issues to make sure that the software will work upstream, and not just in Gentoo (as otherwise there could be nasty surprises, like some I got, where an application works perfectly fine locally, where software is installed through Portage, and fails on Heroku that uses plain Rubygems). You can tell how that can be a PITA by looking at my github page — it lists mostly Ruby packages that I had to “fork” (branch, actually) to get the fixes in; mostly they have been merged upstream, sometimes they are dead in the water though.

All of this makes the situation quite complex; while I sort-of enjoy working with Ruby and these things, I also noted that it takes a very long time to get all the dependency web tested and fixed… and it’s the sort of time that, in my personal free time, I just don’t have. I have been packaging (and thus testing and fixing) a few packages that I triaged for a few job tasks, and some that I’m still using, using the paid work time, but that can’t cut it to work for every package out there. I guess the same thing goes on for Alex, Hans and Gordon.

What’s the bottom-line? Well, Hans in particular has been doing a huge work to port the ebuilds from the old gems.eclass to ruby-fakegem.eclass so that they can be installed when Ruby 1.9 is present without messing that up, even though they wouldn’t work with it. This makes the day that we can get it unmasked much nearer. But there are quite a few cases where we can’t just drop the old version so easily, and it mostly relates to non-gem bindings and the usage of Ruby as a scripting engine (rather than adding support for a library to Ruby itself). And this is without counting further issues like bundler not working altogether too well because it lacks dependency information, or getting Rubygems to refuse messing with the Portage-installed gems altogether (that is now much more feasible than before, since we no longer use the gem command from within Portage to install the stuff).

So what can you do to get this sooner? You can help out by making sure packages work with Ruby 1.9; when they have been positively tested not to work on that version, they are usually marked as so in the ebuild itself; for my part, I always note the problems with an Unicode right-pointed arrow, so running a fgrep command on the tree for “ruby19 →” should give you a very good idea of how many problems (and how many different problems there are out there).

You have no idea where to start with this thing? There is another option: hire me. Well, I would have liked to say “hire us”, but it turns out at least both Alex and Hans are not looking to be hired for this, while a project of mine is delivered this week and then I have some extra time for the next few months. I wouldn’t mind being paid to work full-time on getting Ruby 1.9-ready packages in the tree. I’m a registered freelancer in Italy so I have an European VAT ID and I can make proper invoices, so it’s going to be all clear in the books. If you’re interested you can contact me to discuss pricing and amount of work you’re looking for.

Just please, stop harassing the team because we’re not as fast as you’d like us to be… we’re already doing a hell of a job in a hell of a hurry!

Was macht man eigentlich, wenn der Filius ausgezogen ist, er aber auf dem hauseigenen Server seine Musik, Dokumente, Fotos whatever liegen hat?

Alles auf eine USB Festplatte kopieren ist praktisch, aber nicht gewünscht.

NFS oder SMB über das Internet zugreifbar machen ist nur über VPN sinnvoll. Oder wie hier kurz beschreiben durch tunneln mit ssh.

Erster Schritt ist das forwarding für Port 22 im Router zu aktivieren. Das geht bei unserem Router in der Gui und wird hier nicht weiter beschrieben. Der Knackpunkt ist einfach, das eine ankommende Anfrage für eine ssh Sitzung zum Server weitergeleitet wird. Da man dadurch die Firewall im Router natürlich für alle ssh Anfragen geöffnet hat, sollte man tunlichst fail2ban auf dem Server installiert haben, weil man den Rechner nun angreifbar geben die Brute force Scripte gemacht hat, die laufend die Internet Welt auf schwache Passworte abklopfen.  Fail2ban sperrt nach wenigen Fehlversuchen die IP für 10 Minuten und verhindert somit ziemlich wirksam diese Art von Angriffen. Eine andere Idee wäre, einen beliebigen anderen Port zu verwenden, was ich in der Anfangsphase vermeiden wollte, aber durchaus als zusätzliche Maßnahme empfehle. Natürlich habe ich direkten root Login in der sshd_config verboten, aber das versteht sich ja von selbst.

Der nächste Schritt ist, auf dem Windoof Rechner das Stoppen der evtl. freigegebenen Ordner, weil es unter Windoof nicht anders geht, es sei denn, man creiert virtuelle Hostadapter.

Diesen Weg gehen wir nicht, sondern stoppen also mit net stop server in der Command shell alle möglichen Freigaben.

Hat man sich cygwin installiert, kann man nun ganz normal in der bash den Tunnel via ssh aufbauen.

Ich habe den Befehl mal etwas auseinandergepflückt, weil es mit den Begriffen leicht Verwirrung geben kann.

In ein paar Sätzen beschrieben macht man auf seinen lokalen Rechner, hier also auf dem W$ Rechner des Filius einen Tunnel für das SMB Protokoll auf. Der Tunneleingang ist der lokale Rechner Port 139 und das andere Tunnelende wird als zweiter Parameter in Form einer ganz normalen ssh Syntax angegeben. Die Ip-Adresse ist die des fernen Servers, auf dem man zugreifen möchte.

Der dortige sshd muss forwarding erlaubt haben und kümmert sich um den Rest. Es werden also die TCP/IP Pakete des SMB Protokolls über die SSH Verbindung "getunnelt" und werden im fernen Rechner an den Port 139 weitergegeben.

Zusätzlich ist die sshVerbindung ohne weiteres parallel als ganz normale Login shell benutzbar.

Wenn man es lieber mit putty machen möchte, ist das auch kein Problem. Ich erspare mir hier das, weil es hier genau detailliert erklärt ist.

Hat man den public ssh-key auf den Server kopiert, wird man nun noch nicht einmal nach dem Passwort gefragt.

Wenn man sich nun am Prompt auf einem Server befindet, steht der Tunnel und kann benutzt werden.

Heißt die Freigabe auf dem RemoteServer z.B. server, dann genügt nun ein \\localhost\Server im Explorer und voila!

weitere Quellen:

www.blisstonia.com/eolson/notes/smboverssh.php
hier wird die Sache mit den virtuellen Hostadaptern erklärt.

http://www.ibr.cs.tu-bs.de/kb/samba-ssh.html Hier sind spezielle Infos für das RZ in BS, der allgemeine Teil ist aber zum Verständnis empfehlenswert.

Just like Multimedia Mike, I have been quite sceptic regarding seeing HTML5 as a saviour of the open web. Not only because I dislike Ogg to a passion after having tried to parse it myself without the help of libogg (don’t get me started), but because I can pragmatically expect a huge number of problems related to serve multiple video files variant depending on browser and operating system. Lacking common ground, it’s generally a bad situation.

But I have been hoping that Google’s commitment to support HTML5 video, especially in Youtube, would have given me a mostly Flash-free environment; unfortunately that doesn’t seem to be the case. There is a post on the Youtube API blog from last month that tries to explain users why they are still required to use Flash. On the other hand, it has the sour taste that reminds me of Microsoft’s boasting about Windows Genuine Advantage. I guess that notes such as these:

Without content protection, we would not be able to offer videos like this .

to land me on a page that says at the top “This rental is currently unavailable in your country.” without any further notice, and without a warning that Your Mileage May Vary, makes it very likely to have a mixed feeling about a post like that.

Now, from that same post, I got the feeling that for now Google is not planning on supporting embedded Youtube using HTML5, and relied entirely on Flash for that:

Flash Player’s ability to combine application code and resources into a secure, efficient package has been instrumental in allowing YouTube videos to be embedded in other web sites. Web site owners need to ensure that embedded content is not able to access private user information on the containing page, and we need to ensure that our video player logic travels with the video (for features like captions, annotations, and advertising). While HTML5 adds sandboxing and message-passing functionality, Flash is the only mechanism most web sites allow for embedded content from other sites.

Very unfortunate, given that a number of website, including one of a friend of mine actually use Youtube to embed some videos; even my blog has a post using it. It’s still a shame, because it’s a loss, for Google, of the iPad users.. or is it, at all? I have played around a minute with an iPad at the local Mediaworld (Mediamarkt) last week. And I looked at my friend’s website with it. The videos load perfectly using HTML5 I guess, given that it does not support Flash at all.

So what’s the trick? Does Google provide HTML5-enabled embedded videos when it detects the iPhoneOS/iOS Safari identification in the user-agent? Is it Safari instead to translate the Youtube links into HTML5-compatible links? In the former case, why does it not do that when it detects Chrome/Chromium as well? In the latter, why can’t there be an extension to do the same for Chrome/Chromium?

Once again, my point is that you cannot simply characterize Apple and Google as being absolutely evil and absolutely good; there is no “pureness” in our modern world as it is, and I don’t think that trying to strive for that is going to work at all… extremes are not suited for the human nature, even extreme purity.

The going is tough, but at least there is any going – I think this describes best my progress with Backtory. During the last two weeks, we've been on holiday in Denmark. I used some of the rather rainy days to code on Backtory, and here's what I got done so far:

• Libsynctory is in a working state now, meaning it does what is expected of it. However, it's not yet nice – no proper documentation, no thread safety, no error tracing.
• Libyar got a bit further. The container file layout is finalized and properly documented. An awful lot of macros and some init voodoo for thread saftey are ready, but the lib itself is of no use yet.
• The Backtory application itself hasn't been touched by me for some weeks now – I concentrated efforts on the two above-mentionned libs. However, I have made up my mind about the application (again ). There will be a “cheap” CLI application, most probably implemented in Python. The more complex thing (daemon, configuration via network and stuff) will become eBacktory, of which I'll take care later.

So first priority for me is to get libyar ready to use, and to finish libsynctory into a state that could be called “release-ready”. When both libs are done, I'll first finalize a CLI for YAR files before taking care of the Backtory implementation itself. As I see tough times ahead concerning my professional life, I dare not give any forecast when I'll be able to spare the hours so urgently needed to finish any of the mentionned tasks. But as I already statet – the going is tough, but at least there is any going…

Nach beinahe dreimonatiger Sendepause und vielen Stunden Codewühlerei habe ich meine neue Website für einsatzfertig befunden – und heute live geschaltet. Als Backend kommt nun eine Django-basierte Eigenentwicklung zum Einsatz, welche die einzelnen PHP-Applikationen ablöst.

Hier die wichtigsten Änderungen im Überblick:

• Die URLs im Blog haben sich geändert. In Anlehnung an Wordpress-URLs werden Einträge jetzt immer nach Jahr, Monat und Tag ihrer Veröffentlichung strukturiert und mit einem Slug (Bezeichner) identifiziert.
• Lässt man besagtes Slug in den URLs weg, bekommt man das Blog für das jeweilige Jahr, den Monat oder den Tag angezeigt. Anstelle der Datumsfragmente lassen sich auch einzelne Autoren oder Tags verwenden.
• Die alte Feed-URL http://www.my-universe.com/blog/feeds/index.rss2 ist nicht mehr gültig. Stattdessen funktioniert nur noch http://www.my-universe.com/blog/feeds/ – dafür aber mit derselben Flexibilität wie die Archiv-URLs (Angaben von Datumsfragmenten, Autor oder Tags möglich).

Alle anderen Änderungen betreffen im wesentlichen das Backend und sind daher primär für André und mich relevant.

Eine Bitte noch in eigener Sache: Ich habe mir zwar für die Entwicklung keinen Termindruck auferlegt und auch in meinem Staging-System ausgiebig getestet. Ich will aber nicht ausschließen, dass irgendetwas nicht wie erwartet funktioniert, Links vielleicht tot sind oder eingebundene Bilder fehlen. Wenn so etwas auffällt, bitte eine kurze Mail an mich – dann kann ich zumindest versuchen, es zu reparieren.

• Disable Front Row on Leopard | White snow
  Ein Ansatz, der ohne das Löschen von FrontRow.app auskommt.
  (tags: apple frontrow osx howto)

I can be quite difficult to read for what concerns alternative approaches to the same problem; while I find software diversity to be an integral part of the Free Software ideal and very helpful to find the best approach to various situations, I also am not keen on maintaining the same code many time because of that, and I’d rather have projects to share the same code to do the same task. This is why I think using FFmpeg for almost all the multimedia projects in the Free Software world is a perfectly good thing.

Yesterday, while trying to debug (with the irreplaceable help of Jürgen) a problem I was having with Gwibber (which turned out to be an out-of-date ca-certificates tree), I noted one strange thing with pycurl, related to this fact, that proves my point to a point.

CURL can make use of SSL/TLS encryption using one out of three possible libraries: OpenSSL, GnuTLS and Mozilla NSS. The first option is usually avoided by binary distributions because it is incompatible with some licensing terms; the third option is required for instance by the Thunderbird binary package in Gentoo as it is. By default Gentoo uses OpenSSL, that you like it or not.

When CURL is built against OpenSSL (USE="ssl -gnutls -nss"), PyCURL linked to libcrypto; given that my system is built with forced --as-needed, it also means it uses it. I found it quite strange so I went to look at it; if you rebuild CURL (and then PyCURL) with GnuTLS (USE="ssl gnutls -nss") you’ll see that it only links to libgnutls, but if you look closer, it’s using at least one libgcrypt symbol. Finally if you build it with Mozilla NSS (USE="ssl -gnutls nss") then it will warn that it didn’t detect the SSL library used.

The problem here is that CURL seems not to provide a total abstraction of the SSL implementation it uses, and for proper threading support, PyCURL needs to run special code for the crypto-support library (libcrypto for OpenSSL; libgcrypt for GnuTLS). I’m sincerely not sure how big the problem would be when you mix and match the CURL and PyCURL implementations, I also have no idea what would happen if you were to use CURL with NSS and PyCURL with that (which will not provide locking for crypto at all). What I can tell you, is that if you change the SSL provider in CURL, you’d better rebuild PyCURL, to be on the safe side. And there is currently no way to let Portage do that automatically for you.

And if you are using CURL with NSS and you see Portage asking you to disable it in favour of GnuTLS or OpenSSL, you’ll know why: PyCURL is likely to be your answer. At least once the bug will be addressed.

Matija pointed me at a post about use of CC-NC clause so I wanted to write a couple of notes about my choice of licenses for the two main contents that I actually maintain: this blog and Autotools Mythbuster . Both have the NonCommercial clause added. I have my reasons to do that, and the linked article does not really convince me.

First of all, the blog itself is on a quite restrictive license: BY-ND-NC, NoDerivatives; it means you can copy my posts as long as you attribute them to me, but you cannot do so for a profit, nor you can rework my content into something different. I think it should be mostly fair this way; the best you can think of is the rewriting of my how-tos and other technical articles into proper documentation, and in those cases I usually am the one doing the rewrite, so I can simply write away the ND for that. I don’t see the need to allow reworking of my rants, or of my personal posts.

Also, I find it quite fair that you cannot simply take my posts, even with attribution, push them together others in a personal Planet-like software, add a ton of advertisement units, and profit out of that. Or print my articles as they are in a journal or something and resell that. Without permission that is. The CC licenses are not “my way or the highway”, they only provide the default license of a content, I’m fine with just giving up some clauses if asked, not even nicely, just asked. And note that quoting me is part of the fair use doctrine and does not require abiding to the license per-se.

The other piece of content is the Autotools Mythbuster guide, which is more interesting, and more useful to reuse to write further tutorial and content. At the same time, though, I’d very much like that if anybody felt like publishing (for profit) articles with the use of that content, they’d be asking me about it first; and if somebody were to rework that content as a book, I’m not ashamed to say that I’d like a cut out of it.

Just to point one thing out, to quote the article:

Many bloggers and blog communities on the web use advertising as a way to recoup costs and generate income. Popular bloggers, from Andrew Sullivan to Markos Zúniga (Dailykos), have turned their hobbies into professions, but even smaller publications often use Google Ads to make some extra money. Other sites use small-scale subscription models to unlock additional features and content or disable advertising. Ask yourself if you really want to stop all these individuals from using your work.

I asked myself that and my answer is: “yes, I want to stop those individuals from using my work without asking first.” I don’t think it is pretending too much that if they are not given explicit permission they cannot reuse my content to make a profit out of it, even if they were fored to share it with the same terms (interestingly, the article insists that it’s nigh impossible to enforce NC compliance… but on the other hand repeats that using SA is “enough protection”… talk about mirror climbing).

I remember a few years ago reading about a movement to provide a “pineapple license” (I forgot the name of the thing, I think it had a ‘k’ in there, and that the pineapple was the logo of the license) that stated from one side what the CC license was to everyone, and from the other “screamed” that the author just wish to be contacted for special uses. That would be my preferred solution to be honest. I’m probably fine with reworking my content for many other uses, just ask me about it first, though, please.

What the rest of the article seems to talk about is the use of NC clauses in software and software distributions, making a straw man argument about “what if the kernel was NC-licensed”. I call that a straw man because all of the CCs are vastly unsuited for software, and are at most useful for artwork, documentation and the “ancillary” content of software packages.

It is obvious that a Linux company will be unable to make use of works that prohibit commercial use. But non-profit free software communities are equally adamant in rejecting -NC licenses. For example, the Debian Free Software Guidelines explicitly state: “The license of a Debian component may not restrict any party from selling or giving away the software as a component of an aggregate software distribution containing programs from several different sources.” Debian GNU/Linux is one of the most popular distributions of the open source Linux operating system.

So this shows that NC is unsuited for Debian.. and it’s vastly unsuited for Fedora as well; even the whole of Gentoo documentation is released as Attribution-ShareAlike by itself. But aren’t they now setting up double standards in the article? They actually go a long way to spell the good deeds of the GNU Free Documentation License, which … Debian finds unsuitable for main !

Finally, I’d like to point out one very important thing here: I’m talking about two main content projects that consists almost entirely of texts. While I can understand and agree pretty well with the idea of CC-BY-SA to be enough for other media (graphics, videos, sound, …) texts are a different thing. You can license the content as it’s written, but you cannot license the concepts or the ideas; and while making use of simply the concepts of graphic elements is very difficult and usually you’re just interested in using the content as-is, or rework it further from there, re-using the concept of a text without actually copy-pasting that text is definitely feasible.

Again, can we stop considering the world (and copyright issues) black-and-white? They definitely are not.

Dear Lazyweb:

Ich versuche gerade herauszufinden, was ich genau tun muß, um dem JMStV zu genügen.

Wenn ich alles richtig verstanden habe, dann muß ich bis zum 31.12. alle meine Inhalte, die sich an Deutsche wenden, mit einer Alterskennzeichnung versehen habe. Wenn ich das weiterhin verstanden habe, dann bin ich haftbar, wenn ich zu niedrig einstufe, und muß Geld und Mühe aufwenden, wenn ich per Default alles auf "Ab 18" einstufe (Jugendschutzbeauftragter, Alterskontrolle mit Personalausweis oder Sendezeiten, also Blog tagsüber abschalten).

Das alles auch rückwirkend für Texte, die für mich erledigt sind und die ich nur noch aus Archivgründen und für stabile URLs online halte.

Das bedeutet - falls das alles stimmt - daß ich Ende des Jahres am kostengünstigsten und mühelosesten fahre, wenn ich alle Texte von mir in deutscher Sprache offline nehme, meine Webserver von DE-Domains und Hosting Deutschland weg migriere und die DE-Domain nur noch für Mail verwende, und in Zukunft irgendwo in der Cloud in englischer Sprache blogge.

Ist diese Analyse falsch? Wenn ja, inwiefern?

Nachtrag: Weils gefragt worden ist: Nein, ich habe kein Interesse daran, meine Inhalte Kindern und Jugendlichen zugänglich zu machen. Es geht mir darum, mit minimalen Einsatz von Zeit und Geld maximale Rechtssicherheit für mich herzustellen. Dieses Angebot hier ist kostenlos und werbefrei, von einer Person betrieben. Es darf mich nichts kosten, den Kram hier aufzubereiten und es darf mich nicht rechtlich mehr als notwendig exponieren.

Es geht also darum, für mich mit minimalem Aufwand mit dem neuen JMStV umzugehen - welche Optionen, welche Konsequenzen? Wie werdet Ihr Euer Blog zum 1.1.2011 umstellen?

Do you remember my latest libvirt ranting and the recent post about Kerberos and NFSv4 don’t you? Well, let’s tie the two up and consider a couple of good and bad things related to both.

First of all, as Daniel Berrange pointed out, QEmu does support IPv6; unfortunately it doesn’t seem to work just as he supposed it to: even though my hostname resolves to both IPv4 and IPv6, QEmu by default only listens to v4. The same goes if you don’t provide a listening socket (such as ""), and again the identical same happens with the default setting provided by libvirt (0.0.0.0). You can force it to listen to v6 by either providing a v6-only hostname, a v6 IP address or the v6 catch-all [::] which makes it work on both v6 and v4, lovely, isn’t it?

Then, about libvirt-remote, as many pointed out it is possible to use it with SSH as user, but there are two catches there: the first is that with the way the arguments are passed down from virt-manager down to libvirt, to ssh and zsh on the other side, something goes funky; it works fine with bash because it splits the parameters again, but with zsh as login shell for my user it tries to call a binary called nc -U ... which as you might have guessed is not correct. The second problem is that even if you set the unix socket access for your user, it won’t let it work if you are using SSH and the system is configured with PolicyKit. I guess this was designed to work in two distinct configuration (desktop and server) and trying to mix the two creates a bit of trouble.

This does not solve two problems though: the dangling connections that are kept alive even after closing virt-manager and its inability to provide diagnostic more human-readable than the Python exceptions. This became tremendously obvious today as I went to consider the idea of using Kerberos for the authentication of libvirt itself, given that it can do that via SASL. It would make more sense, since I’ll be having a Kerberos install anyway at this point, to use the Kerberos credentials for more than a couple of services.

Using Kerberos for libvirt actually makes quite a bit of sense: you can set up properly TLS support for the connection and have an user-based authentication (rather than the whole host-based authentication that is supported with the TLS-only login). Setting up libvirt itself is not difficult, if it wasn’t for the single problem that most of the documentation tells you to use /etc/libvirt/krb5.keytab while it’ll be looking only at /etc/krb5.keytab by default — maybe it’s worth for Gentoo to change the init script so that it searches for the one documented. After that, I can properly login on the libvirt-remote access with virt-manager and Kerberos…. but I still am having trouble with QEmu and VNC this time around.

Now a little note regarding pambase: as I’ve been brought to note the default configuration used by pambase with the kerberos USE flag enabled might not be well suited for all the sites using Kerberos right now. I know that, but Gentoo never pretended to give perfect defaults, or defaults that suit everybody; on the other hand I think it’s important to give a default for Kerberos in our packaging. I’ll have to talk with Robin or someone else for integrating a default regarding pam_ldap as well, since the LDAP guide we provide is hinting at the wrong solution for the PAM configuration, if the system also want to be a desktop.

Having found a decent way to provide multiple optional login systems for users is actually finally paving the way to provide token-based login that I talked about last year.

I’ve been fighting the past few days with finding a solution to strengthen the internal security of my network. I’m doing this for two main reasons; from one side, having working IPv6 on the network means that I have to either set up a front-end firewall on the router, or I have to add firewalls on all the devices, and that’s not really so nice to do; on the other side, I’d like to give access to either containers (such as the tinderbox) or other virtual machines to other people, developers for Gentoo or other projects.

I’m not ready to just give access to them as the network is because some of the containers and VMs have still password-login, and from there, well, there would be access to some stuff that is better kept private. Even though I might trust some of the people I’m thinking to give access to, I won’t trust anybody else’s security practice with accessing my system. And this is even more critical since I have/had NFS-writeable directories around the system, including the distfiles cache that the tinderbox works with.

Unfortunately, most of the alternatives I know for this only work with a single user ID, and that means among other things that I can’t use them with Portage. So I decided to give a try to using NFSv4 and Kerberos. I’m not sure if I’ll stick with that sincerely, since it makes the whole system a whole lot more complex and, as I’ll show in a moment, it’s also not really solving my problem at its root, so it’s of little use to me.

The first problem is that the NFSv4 client support in Gentoo seems to have been totally broken up to now, bug #293593 was causing one of the necessary services to simply kill itself rather than running properly, and it was fun to debug. There is a library (libgssglue) that is used to select one out of a series of GSS API providers (either Kerberos or others); interestingly enough, this is yet another workaround for Linux missing libmap.conf and the problem with multiple implementations of the same interface. This library provides symbols that are also provided by the MIT-KRB5 GSS API library (libkrb5_gssapi); when linking the GSS client daemon (rpc.gssd) it has to link both, explicitly causing symbol collisions, sigh. Unfortunately this failed for two reasons again: .la files for libtirpc (don’t ask) caused libtool to actually reorder the linking of libraries, getting the wrong symbols in (bad, and shows again why we should be dropping those damn files), plus there was a stupid typo in the configure.ac file for nfs-utils where instead of setting empty enable_nfsv41 variable they set enable_nfsv4, which in turn caused libgssglue from not being searched for.

The second problem is that right now, as I know way too well, we have no support for Kerberos in the PAM configuration for Gentoo, this is one of the reason why I was considering more complex PAM configurations — main problem is that most of the configurations you find in tutorials, and those that I was proposed, make use of pam_deny to allow using either pam_unix or pam_krb5 at the same time; this in turn breaks the proper login chain used by the GNOME Keyring for instance. So I actually spent some time to find a possible solution to this. Later today when I have some extra time I’ll be publishing a new pambase package with Kerberos support. Note: this, and probably a lot more features of pambase, will require Linux-PAM. This is because the OpenPAM syntax is just basic, while Linux-PAM allows much more flexibility. Somebody will have to make sure that it can work properly on FreeBSD!

There is also a request for pam_ccreds to cache the credentials when running offline, I’m curious about it but upstream does not seem to be working on it as much as it should, so I’m not sure if it’s a good solution.

Unfortunately, as I said, NFSv4 does not seem so much of a good solution; beside the still lack of IPv6 support (which would have been nice to have, but it’s not required for me), if I export the distfiles over NFSv4 (with or without Kerberos), the ebuild fetch operation remain stuck in D-state for the process (blocked on I/O wait). And if I try to force the unmount of the mounted, blocked filesystem, I get the laptop to kernel panic entirely. Now, to make the thing easier to me I’m re-using a Gentoo virtual machine (which I last used for writing a patch for the SCTP support in the kernel) to see if I can reproduce the problem there, and get to fix it, in one way or another.

Unfortunately I’ve spent the whole night working and trying to get this working, so now I’ll try to get some rest at least (it’s 9.30am, sigh!). All the other fixes will wait for tomorrow. On the other hand, I’d welcome thank yous if you find the help on Kerberos appreciated; organisations who would like to have even better Gentoo support for Kerberos are welcome to contact me as well…

Non scrivo spesso in italiano e so che il più delle volte è per lamentarmi, ma uno sfogo di tanto in tanto fa bene direi. Oggi appunto vorrei lamentarmi della società che si occupa della gestione di rifiuti e acquedotto nella maggior parte della provincia di Venezia, compreso dove abito io, ovviamente: Veritas.

Lo scorso ottobre, per una serie di ragioni troppo lunghe da spiegare, ho richiesto il subentro come cliente, prendendo il posto dei miei genitori, a cui erano intestate, separatamente, le bollette di acqua e rifiuti TIA. Ho anche deciso di richiedere contestualmente l’addebito bancario diretto, visto che altrimenti sono le solite opzioni: pagare con bonifico (€1 a colpo) o con bollettino postale (€1.2 e un viaggio fino alle poste che ovviamente sono aperte solo di mattina, o anche di più se si vuole pagare con carta di credito dal sito delle poste, sempre che funzioni).

Ad ogni modo, la richiesta viene presa in carico, e me ne rendo conto quando rientrano i soldi della cauzione pagata da mio nonno (l’originale intestatario della bolletta dell’acqua quando ancora si parlava di Aspiv). Anche dal sito della mia banca noto l’accettazione della domiciliazione bancaria sui pagamenti utenze, quindi vado tranquillo.

La prima bolletta arriva a Marzo, in cui risulta essere domiciliata, anche se mancano i dettagli bancari sulla fattura stessa, ma imputo la cosa a un errore di stampa, e la seconda bolletta pure. Finché un giorno non mi ritrovo una minacciosa lettera da Veritas che mi intima di pagare le due bollette arretrate. Ohibò! Controllo ed effettivamente il pagamento non è mai stato effettuato. Chiamo, la mattina successiva, per scoprire qual’è il problema. “Lei non ha fornito l’IBAN” “Guardi sono abbastanza sicuro di averlo fatto, ho qua la documentazione” “No guardi che se l’avesse fornito l’avremmo inserito quindi non l’ha fornito”.

Dopo aver provato a far capire alla non-proprio-gentile operatrice che se la mia banca mi indica che la domiciliazione è inserita, che il loro stesso sistema non mi fa ricevere i bollettini postali di pagamento, e che le fatture riportano che ho richiesto la domiciliazione, significa che c’è un problema a monte, decido di lasciar perdere e chiedo se può inserire lei l’IBAN per me. No, devo inviare un fax (nel 2010!)… oppure utilizzare lo Sportello Online. Oh bene, almeno qualcosa che funzioni, si spera. Effettivamente dal loro sito risulto mancare l’IBAN; visto che l’IBAN è diventato necessario per le domiciliazioni da non molto mi aspetto che sia un avviso apposito, e compilo il mio IBAN senza dubbi ulteriori.

Finché ieri pomeriggio non mi arriva una nuova fattura di Veritas, che mi fa notare che la precedente fattura non è stata pagata. E ancora una volta riporta la dicitura “_Secondo le disposizioni da Lei impartite, l’importo sarà addebitato salvo buon fine presso:
con codice ABI/CAB /._” . Testuali parole, si è tutto lasciato in bianco.

Oggi chiamo di nuovo, e l’operatrice, un po’ più gentile, ma comunque stizzita quando ripeto che il problema è loro mi fa sapere che comunque a loro non risulta alcuna richiesta di RID da parte mia (e allora perché non ricevo neanche il bollettino? Mah!), e che l’unica cosa che posso fare è scaricare il modulo dal sito (che insiste nel ridarmi due volte anche se le ho ripetuto che stavo guardando lo Sportello Online), e compilarlo comprensivo di documento d’identità, e spedirlo, ovviamente, via fax.

Nel frattempo, ho da pagare due bollette di Veritas tramite bonifico bancario, cosa che mi scoccia già che basta. Perlomeno non sembrano aver inserito interessi di mora per il pagamento in ritardo (perché se fosse successo, in quel caso il fatto che il problema fosse loro non sarebbe decisamente passato in secondo luogo così facilmente).

E tutto questo perché? Perché probabilmente qualche impiegato ha ben voluto ignorare il fatto che esiste l’IBAN e ha compilato la mia pratica con solo ABI e CAB nel momento del subentro, che da un lato è bastato per effettuare la richiesta di domiciliazione, ma dall’altro, essendo la prima fattura stata emessa nel 2010, non era abbastanza per effettuare l’addebito.

Poi il problema è che i giovani italiani non vanno fuori di casa, eh? Chissà come mai.

Apple verkündet auf der Antennagate Pressekonferenz, daß sie drei Millionen iPhone 4 in weniger als 3 Wochen verkauft haben, das sind eine Million Telefone pro Woche oder im Mittel die 150.000 pro Tag. Genau genommen sind 1.7 Millionen in den ersten drei Tagen verkauft worden, und weitere 1.3 Millionen in den folgenden Wochen. Wenn man das als Grundlage heranzieht, kommt man auf knapp 75.000 Einheiten am Tag Dauerverkaufsleistung.

Auch Google hat Grund sich zu freuen, denn jeden Tag gehen 160.000 Android-Phones neu online, also laufend etwa 2x mehr als iPhones pro Tag, und dauerhaft etwa gleich viel wie das iPhone als Spitzenverkaufszahlen erreicht.

Diese Zahlen liegen aber nach anderen Quellen noch immer unter den Zahlen von Nokia, die im 4Q2009 20.8 Mio Smartphones verkauft haben sollen, also etwa 230.000 pro Tag. RIM/Blackberry wird mit 117.000 Exemplaren pro Tag beziffert, ist also etwa eineinhalb mal so stark vertreten wie Apple.

Das eigentlich schockierende aber ist ein Artikel bei Business Insider, der deutlich macht, wie überlegen das Geschäftsmodell von Apple ist: "Apple will generate 2X as much handset profit as the rest of the industry combined this year despite selling only 3% of the handsets by volume." Das wiederum sollte vor allen Dingen den Netzbetreibern zu denken geben, denn aus deren Tasche muß das Geld ja wohl kommen - die Gerätepreise sind nicht so unterschiedlich.

Nachdem sich ja auch Apple endlich entschlossen hat, ihre Methode der WLAN-Kartierung auf sanften Druck hin offen zu legen (abschalten) relativiert sich die Aufregung um Google vielleicht ein wenig.

Bei Dennis Morhardt findet man das interessante Konzept des Datenausweises. Das W3C (und Datenschutzzentrum.de in Kiel!) war seiner Zeit weit voraus, als es genau diese Idee vor sieben Jahren schon unter dem Stichwort P3P ("Platform for Privacy Preferences") ausformuliert und standardisiert hat.

Leider war der öffentliche Druck damals nicht groß genug, sodaß kein Hersteller die P3P-Kernstandards jemals implementiert hat (Microsoft hat immerhin die P3P Kurzversion in ihren Cookie Preferences umgesetzt, die anderen Hersteller haben weiter gepennt), und der Standard muß sicher auf die Bedürfnisse der aktuellen Debatte leicht angepaßt werden, ist aber genau das, was für den Datenausweis (und den ePerso!) gebraucht wird.

Als Antwort auf die Frage der Netzwerkneutralität hat die Telekom schneller reagiert als sie einen DSL-Anschluß legen kann und ein Antwortvideo produziert.

(Youtube DirektTelekom)

Die TL;DS (Too Long; didn't see) Zusammenfassung: Ihr kriegt so viel Netzwerkneutralität, wie ihr bezahlen könnt. Das Entlarver-Wort im Interview: "Basis-Internet". Die Begriffe Peering und Transit werden sauber aus der Debatte herausgehalten, stattdessen ist von "Milliardeninvestitionen" die Rede.

Was wirklich passiert ist, ist daß der deutsche Staat dem Privatunternehmen Telekom die gesamte Telekom-Infrastruktur Deutschlands geschenkt hat, als die Post privatisiert worden ist und daß die Telekom jetzt die Eyeballs am Ende der letzten Meile in Geiselhaft nehmen will.

Netzpolitik verweist auf Golem verweist auf die Frankfurter Rundschau. Die deckt auf: DE-Mail ist nicht sicher, weil es nicht durchgehend verschlüsselt, sondern "Auf den Servern jedoch werden die Mails aus technischen Gründen kurz entschlüsselt und sofort wieder verschlüsselt".

Das muß wohl auch so sein, jedenfalls wenn man Antivirus und Antispam implementieren will, insofern scheint die Lösung erst mal grob legitim zu sein, auch wenns komisch klingt.

Komisch klingen tut es aber noch aus einem anderen Grund: Die Vertraulichkeit ist im Kontext DE-Mail nicht primär relevant, denn DE-Mail war nie designed um sicher im Sinne von vertraulich zu sein. Es geht stattdessen um die Schaffung von Rechtssicherheit und das ist im wesentlichen keine technische, sondern eine juristische Konstruktion, bei der die technischen Maßnahmen nur qualifizierende Begleiterscheinungen sind, die die juristische Fiktion von DE-Mail stützen sollen. Daher laufen Verweise auf PGP und ähnliches auch am Thema vorbei.

Das entscheidende Merkmal ist nicht die technische Konstruktion von DE-Mail, sondern das DE-Mail Gesetz (PDF, Entwurf), das festlegt, daß auf Grund der besonderen technischen Konstruktion von DE-Mail besondere juristische Regeln wirksam werden. Diese stellen eine DE-Mail dann je nach Versandart einem Brief oder einem eingeschriebenen Brief oder einem eingeschriebenen Brief mit Empfängeridentifikation und Rückschein gleich. Das passiert aber nicht wegen der tollen Technik, sondern auf der rechtlichen Grundlage des noch zu schreibenden DE-Mail Gesetzes - für einen Techniker ist das juristischer Schamanismus, weil die technischen Garantien des Verfahrens aus der technischen Sicht eben genau das nicht leisten, was das Gesetz dann anordnet.

Für den Kunden/Endnutzer sind auch andere, juristische und nur am Rande technische Überlegungen wichtig, aus denen man DE-Mail wahrscheinlich eher nicht nutzen will.

Merkel so:

Heute wird es durch die Vielzahl der Informationskanäle, und besonders durch das Internet, immer schwieriger, ein Gesamtmeinungsbild zu erkennen. ... Mit dieser Veränderung muss die Demokratie in Deutschland und in den anderen westlichen Ländern umgehen lernen.

Und redet auch sonst von 'reagieren'. Denn das agieren, die Konzepte und das Gestaltungsprimat der Politik, das sind in Deutschland systematisch schon lange keine Themen mehr - dazu hat man sich ja auch mutwillig viel zu sehr in die Ecke gemalt.

Abgeordnetenwatch so:

Merkel: Politiker müssen Umgang mit Internet erst noch lernen (OpenReport).

Das stimmt: Merkels Statistik auf Abgeordnetenwatch: 96 Fragen, 0 Antworten.

Karpfenpeter verliert ein Zyn.

Doom II - laut Wikipedia eingestuft als 18+/Indexed bei der USK. Auf dem PC. Genau wie Doom III.

Dasselbe Doom II in feinem 1024x768 auf dem iPad im iTunes Store.

Und hier warum das alles so ist. Logik hat das keine mit dem JMStV - nicht im alten und auch nicht im neuen.

In einer Diskussion neulich meinte jemand zu mir: "Man müßte einfach den ganzen Demonstrationszug mit Kameras sättigen. Wenn es dann zu Polizei-Übergriffen kommt, gibt es einfach so viele Bilder, daß auf jeden Fall genug Material vorhanden ist, um die zweifelsfrei zu identifizieren, egal ob die Nummern haben oder nicht. Vielleicht reicht es sogar dazu, den Tathergang in 3D zu rekonstruieren."

Das geht.

Ein Kollege von mir hat zehn Kameras (#3 Type) für neun Dollar das Stück gekauft (eBay, direkt), und die Dinger sind wohl hackable, bzw wohl sogar hackpflichtig, wenn man bestimmte Modifikationen will.

Für ein wenig mehr Geld gibt es 720p in einem etwas dickeren Gehäuse von einem anderen Anbieter. Ein damit dokumentierter Selbstmordversuch als Einsatzbeispiel:

Die Telekom stellt sich laut dem Heise Newsticker offen gegen Netzwerkneutralität. Das Kostenargument, das sie anführt, ist dabei ein Nullargument - Diensteanbieter und Konsumenten bezahlen jeweils ihren Provider, der Datentransfer ist also schon einmal doppelt bezahlt. Jetzt noch einmal auf der Empfängerseite den Sender abzukassieren ist mehr eine 'Weil es geht'-Aktion.

Netzwerkbetreiber untereinander kassieren voneinander auch schon, wenn die Datenvolumina nicht in etwa ausgeglichen sind. Dabei hat man dann entweder ein Peering-Abkommen oder zahlt für den Transit. Und betreffend datenintensive Dienste wie Youtube lese man bitte einmal Brough Turner durch und verstehe, was dort gesagt wird. Aber unsere Politiker sind für so etwas nicht internetfest genug, daher die 'Weil es geht'-Aktion.

Der andere 'Weil es geht'-Faktor ist die Anzahl der Benutzer, deren letzte Meile die Telekom gefangen hält, und an die Diensteanbieter natürlich ran wollen. Allerdings ist 'Dann schalten wir ihnen Youtube und Google ab' auch eine Art Selbstmord-Drohung, auf mehr als eine Weise.

Vor drei Jahren schrieb ich etwas zu Echten Helden und daß das Wunder unserer Zeit darin liegt, daß es leichter denn je ist, ein Held zu werden, indem man einfach etwas tut und dadurch die Welt fundamental verändert.

Warum wollen Leute Echte Helden sein? Dazu gibt es ein wunderbares Video von RSAnimate, das erklärt, wieso Geld als Motivator nicht funktioniert außer für die primitivsten und langweiligsten Tätigkeiten und was Leute wirklich motiviert: Self Direction, Mastery und Purpose. Also die Freiheit, das eigene Umfeld selbst bestimmen und kontrollieren zu können, und der Wunsch, etwas sinnvolles und wichtiges zu tun und darin und den dazu notwendigen Fertigkeiten besser zu werden. Dahinter stehend also genau der Wunsch, die Welt in einen besseren Ort zu verwandeln.

Anders gesagt: Glück und Heldentum gehen zusammen, wir sind geschaffen, Echte Helden zu sein - es ist in uns eingebaut und es ist das, was wir alle tun wollen, sobald die direkten Existenzsorgen vom Tisch sind.

Beside the lack of up-to-date and sane documentation about autotools (for which I started my guide that you should remember is still only extended in my free time), there is a second huge user experience problem: the skeleton build system produced by the autoscan script.

Now, I do understand why they created it, the problem is that as it is, it mostly creates fubar’d configure.ac skeletons, that confuse newcomers and causes a lot of grief to packagers and to users of source-based distributions (and those few who still think of building software manually without getting it from their distribution).

The problem with autoscan is that it embodies again the “GNU spirit”, or actually the GNU spirit of the original days, back when GNU tried to support any operating system, to “give freedom” to users forced to use those OSes rather than Linux itself. Given that nowadays FSF seems to interest itself mostly on discouraging anybody from using non-free operating systems (or even, non-GNU based operating systems) – sometimes failing and actually discouraging companies from using Free Software altogether – it seems like they had a change of ideas in the middle of that. But that’s something for another post.

Anyway, assuming that you’ll have to make your software work on any operating system out there is something that you are well unlikely to do. First of all, a number of projects nowadays, for good or bad, target Linux only; sometimes even just GNU/Linux (that is they don’t support running on other C libraries) because they require specific features from the kernel, specific drivers and other requirements like that. Secondly, you can easily require your users to have a sane environment to begin with; unless you really have to run on a 15 years old operating system, you can assume at least some basic standard support. I have already written about pointless autotools checks but I guess I didn’t make it too clear yet.

But it’s not just the idea of just dropping support for anything that does not support a given standard, whatever that might be (C99, POSIX.1-2008, whatever), it’s more that the configure.ac generated by autoscan is not going to make it magically work on a number of operating systems it didn’t support before. What it does, for the most part, is adding a number of AC_CHECK_HEADERS and AC_CHECK_FUNCS calls, which will verify the presence of various functions and headers that your software is using… but it won’t change the software to provide alternatives; heck there might not be alternatives.

So if your software keeps on using strings.h (which is POSIX) and you check for it in configure phase, you’re just making the configure phase longer without any solution, because you’re not making use of the results from the configure phase. Again, this often translates to things like the following:

#ifdef HAVE_STRINGS_H
#include <strings.h>
#endif

Okay, so what is the problem with this idea? Well, to begin with, I have seen it so many times without an idea of why it is there! A number of people expect that since autoscan added the check, and thus they have the definition, they have to use it. But if you use a functiont hat is defined in that header, and the header is not there, what are you going to do? Not including it is not going to make your software any more portable, if anything you’re going to get an implicit declaration of the function, and probably fail later at runtime. So, if it’s not an optional header, or function, just running the check and using the definition is not enough.

A common alternative is to fail the configure step if the header or function is not found, while it makes a bit more sense, I still dislike that option, Sure you might be able to tell the reason why the function is needed and whether they have to install something else or upgrade their system, but in truth that made much more sense when there was near to no common ground between operating systems, and users were the common people running the ./configure script. Nowadays, that’s a task that is often limited to packagers, that know their systems much better. The alternative to failing in configure is failing during build, and it’s generally not too bad. Especially since you’ll be failing build for any condition you didn’t know about beforehand.

I have another reason to provide, as for why you shouldn’t be running all those tests for things you don’t support a fallback for: autoconf provides means to pass external libraries and include directives to the compiler; since having each package provide its replacement for common function is going to cause a tremendous amount of code duplication (which in turn may cause a lot of work for packages if one of them is broken, such as dtoa() anybody remembers that?), I’m always surprised that there aren’t many more libraries that provide compatibility replacements for the functions missing in the system C library (gnulib does not count as it’s solving the problem with code replication, if not duplication). Rather than fail, or trying to understand whether you can build or not depending on the OS used, just assume their presence if you can’t go without, and leave it to the developers running that system to come up with a fix, which might involve additional tests, or might not.

My suggestion here is thus to start considering first the operating systems you’re targeting directly; try to find what actually changes between them; in most cases, for instance, you might have still pieces of very-old systems around, like the include for malloc.h that is only useful if you want to call functions such as memalign() but is not used for malloc() since, well, ever (stdlib.h is enough for that), and that will cause errors on both FreeBSD and OS X if included. So once you find that a header is not present in some of your desired operating system, look up what replaces it, then make sure to check for it properly; that means using something like this:

dnl in configure.ac
AC_CHECK_HEADERS([stdint.h inttypes.h], [break;])

/* in your C code */
#if HAVE_STDINT_H
#  include <stdint.h>
#else HAVE_INTTYPES_H
#  include <inttypes.h>
#endif

This way you won’t be running checks for a number of alternative headers on all systems: most modern C99-compatible system libraries will have stdint.h available, even though a few older systems will need for inttypes.h to be discovered instead. This might sound cheap, since it’s just two headers, but especially when you’re looking for the correct place of a library header, you might end up with an alternative among three or four headers, and add a bunch of alternatives here, and you’re going to have problems. The same trick can be used for functions, and the description is also on my guide so and I’ll soon expand it to cover functions as well.

It shouldn’t “sound wrong” to have a configure.ac with near to no AC_CHECK_* call at all! Most of the tests autoconf will do for you, and you have the ability to add further, but there is no need to strain at using them when they are unneeded. Take as example the feng configure.ac — it has a few checks, of course, but they are limited to a few optional features that we workaround if missing, in the code itself. And some I would probably just remove (like making ipv6 support optional… I’d sincerely just make it work if it’s found on the system, as you still need to enable it in the configuration file to use it anyway).

And please, please, just don’t use autoscan, from now on!

Mein diesjähriger Networkers-Rucksack ist jetzt beim ersten Gebrauch nach der Networkers kaputt gegangen. Damit hat er auf jeden Fall länger gehalten als die Rucksäcke etlicher Kollegen, die schon in der Woche der Networkers zerfetzt auseinander gefallen sind.
Tja, “qualitativ hochwertig” war dieser Rucksack wirklich nur auf den ersten Eindruck.

Der CCC ergreift das Wort und gibt ein paar grundlegende Statements zur aktuellen Netzpolitik-Diskussion heraus - sehr gut!

Wir haben daher in klare Worte gefaßt, welche Errungenschaften erhalten und welche aktuellen Mißstände unserer Meinung nach angepackt werden müssen, welche Risiken für die Zukunft einer wettbewerbs- und lebensfähigen Gesellschaft im Netz wir sehen und wohin die Reise gehen soll. Diese Reise kann natürlich nur unter Mitnahme aller Bürger, die ausreichend schnell, unzensiert und unbevormundet an ein interaktives Netz angeschlossen sind, Fahrt aufnehmen.

Den ganzen Artikel könnt ihr hier lesen...

(via: fefe)

CCC: Forderungen für ein lebenswertes Netz is a post from: nodomain.cc

Related posts:

1. Anarchie im Netz
2. 5000 Hits1!11elf1eins!!
3. Dieter Bohlen nackt

In Transparenz - "Das Internet vergißt nicht" - Gut so! schreibt Martin Weigert:

Als ich zwischen 2003 und 2006 mein Bachelorstudium in Wirtschaftskommunikation absolvierte, war die (Werbe-)Welt noch in Ordnung. Die Grundregel war, dass man mit der richtigen Positionierung und Wahl der Kommunikationskanäle im Prinzip jede Botschaft rund um ein Produkt oder eine Dienstleistung vermitteln konnte.

...

Und heute? Jede Aussage, jedes Produktversprechen und jedes kommunizierte Alleinstellungsmerkmal ist googlebar und lässt sich mit Informationen von Bewertungsplattformen, Onlineshops mit Bewertungsfunktion, Blogs, Twitter- und Facebook-Erwähnungen verifzieren.

...

Immer häufiger Frage ich mich, wer eigentlich noch all die mangelhaften Produkte kauft, die als das Beste vom Besten angepriesen werden, aber diesem Anspruch in keiner Weise gerecht werden – was sich natürlich überall im Netz nachlesen lässt.

Er schlägt dann den Bogen von der Werbe-Wirtschaft zur PR und zur Politik, und wie sich dort das Handeln an die neue durch das Netz bewirkte Transparenz anpassen muß.

Im Großen und Ganzen kann ich seinen Thesen zustimmen, aber haken tut es bei mir, wenn er schreibt:

Fehler macht jeder. Wirklich unpopulär werden sie erst, wenn der Eindruck entsteht, dass sie vertuscht oder in ihren Ausmaßen geringer dargestellt werden sollen, als sie tatsächlich sind.

Und da greift der Artikel zu kurz.

10 Jahre “Kölner Lichter” – was zum Geier ist das, dachte ich mir. Ansgar erzählte mir davon und auch Michael (einer meiner Feuerwehr-Kameraden) schwärmte förmlich davon. Er ist extra aus Hamburg nach Köln gereist – nicht nur um dieses Event zu besuchen, sondern vielmehr seine Freundin zu sehen, welche in Köln lebt.

Wir trafen uns also zwischen Eisenbahnbrücke (am Dom) und dem Rheinpark auf Deutzer Seite des Rheins. Michael hielt wirklich sehr gute Plätze direkt an der hochgelegenen Flutmauer frei und wir konnten ein tolles Programm mit Feuerwerk erleben.

Punkt 23.30 ging es mit dem großen Feuerwerk los.

I know that Apple got a lot of hate from Free Software developers (and not) for the way they handle their App Store, mostly regarding the difficulty to actually get application approved. I sincerely have no direct experience on the thing, but if I apply what I learnt from Gentoo, the time they might take to get the applications approved sounds quite about right for a thorough verification.

Google on the other hand, was said to take much less time, but by personal experience to search for content on the Android Market, I can only find DVD Jon’s post quite on the line. There are a number of applications that are, if not entirely, on the verge of frauds, that got easily approved.

On the other hand, as soon as Google was found to add to the Froyo terms of services the fact that they reserve the option of remotely killing an application, tons of users cried foul. Just like they did for Apple, that also has the same capability and has been exercising it for applications that were later found not to agree with their terms of services.

A note here: you might not like the way Apple insists on telling you what you should or should not use. I understand it pretty well, and that’s one of the reasons why I don’t use an iPhone. On the other hand, I don’t think you can say that Apple is doing something evil by doing so. Their platform, their choice; get a different platform for a different choice.

So there are a number of people who think that Apple’s policy in reviewing application is evil (and Google’s allowing possible frauds is a-ok), and in both cases, the remote killswitch is something nasty and a way for them to censor the content for whatever evil plan they have. That points a black light on both of them, doesn’t it? But Mozilla should be fine, shouldn’t it?

I was sincerely wondering what those people who always find a way to despise “big companies” like Apple and Google at the same time, asking their users to choose “freer” alternatives (often times with worse problems) would think while I was reading Netcraft’s report of the malware addon found on the Mozilla index.

I quote: “Mozilla will be automatically disabling the add-on for anyone who has downloaded and installed it.” So Mozilla has a remote killswitch for extensions? Or how are they achieving this?

And again: “[Mozilla] are currently working on a new security model that will require all add-ons to be code-reviewed before becoming discoverable on addons.mozilla.org.” Which means they are going to do the same thing that Apple and Google already do (we’ll have to wait and see to find out to which degree).

Before people misunderstand me: I have nothing against Mozilla and I think they are on the right track here. I would actually hope for Google to tighten their approval process, even if that means much longer turnaround for new applications to be available. As an user, I’d find it much more reassuring than what we have right now (why half the demo/free versions of various apps want to access my personal data, hmm?).

What I’m trying to say here, is that we should really stop crying foul for any choice that Apple (or Microsoft, or Sony, or whoever) makes, they might have quite good reasons to do so, and we might actually follow their steps (like Mozilla appears to be going to do).

Ich hab mir vor ein paar Tagen im ThinkPad-Forum eine kaum gebrauchte Intel X25-M SSD-Platte gekauft, da ich mich einfach mal mit dem Thema Solid State Drive befassen wollte und ich bisher fast nur unglaubliche Erfolgsgeschichten dazu gehört hab. Ergo stieg der “Must have”-Faktor in letzter Zeit immer mehr an und ich suchte aktiv nach einer günstigen Möglichkeit an so ne Platte zu kommen.

Heute kam die Platte an und ich hab sie flugs in mein X61s transplantiert, darauf ein frisches Squeeze installiert und ich komme aus dem staunen nicht mehr raus. Vom Zeitpunkt des Einschalten des Notebooks bis zum grafischen Login (slim) vergehen nur knapp 24 Sekunden. Das BIOS verbraucht davon einen nicht unerheblichen Teil. GNOME startet ab Kennworteingabe in 5 Sekunden. Iceweasel ist in 2-3 Sekunden da, Chromium in geschätzt einer Sekunde. Auch OpenOffice.org ist wesentlich fixer als früher geladen. Ich bin hin und weg. Ich wollte das Notebook schon fast aufs Altenteil schieben, weil es quai zusehens immer langsamer wurde. Ich hatte schon auf das ThinkPad X200s geschielt. Wenn nichts dazwischenkommt wird mir so mein geschätztes X61s mit 4:3-Display (quasi das letzte seiner Art *g*) noch lange treue Dienste leisten.

Einen Teil des Geschwindigkeitszuwachses schreibe ich der frischen Installation zu; meine alte Installation hat schon ein wenig Speck angesetzt, sie ist ja auch schon über mehrere Rechner und Festplatten immer mal umgezogen und ich würde sagen schon über 6 Jahre alt.

Zwar hat die neue Platte “nur” 80 Gigabytes, vergleichbar wenig, wenn ich die 250 GB-Platte vorher dagegen stelle, aber irgendwas ist ja immer.

… zwischen der WM-Mannschaft, die den zweiten, und die den dritten Platz gemacht hat?

Die Spieler der zweitplatzierten Mannschaft hat sich direkt nach der Rückkehr aus Süd-Afrika von mehreren hunderttausend Fans in Amsterdam feiern lassen. So viele Fans waren zumindest erwartet, was genau wie zum Finale zu einem enormen Verkehrschaos geführt hat. Auf den Strassen kommt diese Zahl wohl auch hin, und auf dem Wasser sah es alle paar Meter so aus:

Unsere Elf war nach der Ankunft ja eher miesepetrig.

Since my original post from two years ago didn’t reach yet all the users, and some of the developers as well, I would like to reiterate that you should not be enabling ccache unconditionally.

It seems like our own (Gentoo’s) documentation is still reporting that using ccache makes build “10 to 5 times faster”. I’ll call this statement for what it is: bullshit. The rebuild of the same package might have such a hit, but not the normal emerge process of a standard user with Gentoo. If anything at all, the use of ccache will slow your build down, and even add further failure cases and make it difficult to identify errors.

Now, since the approach last time might not have been clear enough, let me try a different one, by describing the steps it takes when you call it:

• it has to parse the commandline to make sure you’re calling it for a single compile, it won’t do any good if you’re using it to link, or to build multiple source files at once (you can, especially if you use -fwhole-program, but that’s for another day to write about), so in those cases, the command is passed through to the compiler itself;
• once it knows that it’s doing a single compile, it changes the call to the compiler so that instead it simply preprocess the file, and stores the result in a temporary area;
• now it’s time to hash the data, with md4 (the parent of MD5), that as the man page suggests is a strong hash; this has good reasons to be strong, but it also means that it takes some time to hash the content; we’re not talking about the source files themselves, that are usually very small and thus quick to hash, but rather of the preprocessed file, which includes all the headers used… a quick example on my system, by just including eight common header files, produces a 120KB output (with -O2 and _FORTIFY_SOURCE… it goes down to 93KB if -O0 is used); to that add the extra information that ccache has to save (check the man pages for those);
• now it has to search the filesystem, within its cache directory, if there is a file with the same md4; if there is, it gets either copied (or experimentally hardlinked, but let’s not go there for now), otherwise the preprocessed file is compiled and copied in the cache instead; in either case, it involves copying the object file from one side to the other.

Now, we can identify three main time-consuming operations: preprocessing, hashing and copying; all of them are executed whether this is a hit or a miss; if it’s a miss you add to that the actual build. How do they fare about the kind of resources used? Hashing, just like compiling, is a CPU-intensive operation; preprocessing is mixed (you got to read the header files from around the disk); copying is I/O-intensive. Given that nowadays most systems have multiple CPU and find themselves slowing down on I/O (the tinderbox taught me that the hard way), the copying of files around is going to slow down the build quite a bit. Even more so when the hit-to-miss ration is high. The tinderbox, when rebuilding the same failing packages over and over again (before I started masking the packages that failed at any given time), had a 40% hit-to-miss ratio and was slowed down by using ccache.

Now, as I already wrote, there is no reason to expect that the same exact code is going to be rebuilt so often on a normal Gentoo system… even if minor updates to the same package were to share most of the source code (without touching the internal header files), for ccache to work you’d have to leave untouched compiler, flags, and all the headers of all the dependent libraries… and this often includes the system header files from linux-headers. And even if all these conditions were to hold true, you’d have to have rebuilt object files for a total size smaller than the cache size, in-between, or the objects would have had expired. If you think that 2GB is a lot, think again, if you were to use -ggdb especially.

Okay now there are some cases where you might care about ccache because you are rebuilding the same package; that includes patch-testing and live ebuilds. In these cases you should not simply set FEATURES=ccache, but you can instead make use of the per-package environment files. You can then choose two options: you can do what Portage does (setting PATH so that the ccache wrappers are found before the compilers themselves) or you can simply re-set the CC variable, such as export CC="ccache gcc". Just set it in /etc/portage/env/$CATEGORY/$PN and you’re done.

Now it would be nice if our terrific Documentation team – instead of deciding once again (the last time was with respect to alsa-drivers) that they know better what the developers should support – would understand that stating in the handbook that ccache somehow magically makes normal updates “5 to 10 times faster” is foolish and should be avoided. Unfortunately upon my request the answer hasn’t been what you’d expect from logic.

I have written recently about my standing on anti-corporate feelings and I have written a longer time ago speaking against ‘pirate’ software but today I feel like ranting a bit about the way the FLOSS people who still call proprietary software “illegitimate” are hurting the whole cause.

It so happens that a situation like the one I’m going to describe happened to me with more than a couple prospective clients. With one variation or another, but the basic situation is more or less the same.

I get called up by the prospective customer, that is looking for some kind of software solution, or mixed software-hardware solutions, they present me their need, and after a bit of thinking about it, I find there are two solutions: use Free Software, but usually requires fiddling with set-up, tweaking, and maintenance, or use a proprietary solution, with a high license code but a smaller requirement for set-up or maintenance.

I usually present the pricing together with a pros/cons fact sheet, pointing out that whatever proprietary solution will rely solely and exclusively on the original vendor, and thus the first time that vendor does something that goes against your wishes or necessities, you’re left with paying for something you can’t make good use of. While this is usually something that is not easily forgotten, they are scared by the price.

I do my best to provide with options that are cheaper than the license of the proprietary software, so that there is a better chance for the Free alternative to be picked up. It’s not difficult given most of the problems I’ve been shown are solved by proprietary software that is very expensive. Also, it is in my personal interest to have them choose the Free Software solution: I get the money and I usually can release at least the fixes (or even better, the customisation) as Free Software, thanks to licenses such as GPL.

But here, most of the hopes get shattered: “You call it Free but we have to pay quite a bit of money for it… we can get the other cheaper, just use eMule”. At least here in Italy, honesty is a rare virtue, too rare a virtue for it to be “exploited” by Free Software. But why do I say that it’s a mistake of FLOSS developers of this is the case? Isn’t it just the doing of a business holder who cares not about legality and using unauthorized copies? Well, yes of course.

On the other hand, talking about illegitimacy and immorality of proprietary software, defending “piracy” (or unauthorized copies if you wish to call them that way), does not really help the cause, it actually gives them arguments such as “well, but even the guys developing that stuff defend using cracked copies of software, why should I pay you to create something anew when there is the program already?”.

As I said before, make sure the people around you understand why they should use Free Software, and that is not by telling them how bad copy-protection is, DRM is, and the “sins” of Windows. It’s by showing them that they have a price to pay to use that software both in direct monetary terms and in flexibility. And maybe more money would flow into the pockets of the Free Software developers that can make it not suck in the areas it currently sucks.

I’m no Electrical Engineer, I’m no expert in the field of Energy policies and so on, but I really got to rant a bit about the current situation I face. Currently, while not technically living alone yet, I am left to provide for the house; for this reason I’m caring even more about the various bills and prices, and hardware and household equipments.

A few years ago, in Italy, most of the power meters with electronic counterparts capable of reporting instant power consumption to the power company (that for the most part is still provided by the ex-state power company). About at the same time, they started providing a “free market” alternative billing, that provided discounted power for the night (or the weekend!), telling people that, after all, they consumed more power when they were at home.

True as that might be, and efficient as most of the household equipment is nowadays, a few consumer-protection organisations did enough calculations to show that it was almost a scam. It is true that there is more consumption when you’re at home than not, but it is not like there is no power consumption when you’re not home. Even if you were to properly turn off all the equipment that would otherwise go into standby (TVs, stereo, amplifiers, …) you still got a lot of things that are designed not to be turned off, ever (fridges and freezers, answering machines, programmed TV recorders), and a number of things that you’d leave on anyway (like PCs downloading stuff from P2P — there’s no denying that for almost all cases!). While there was a high discount for the price of energy during the evening, it couldn’t cover the huge increase in price during the day.

Luckily, my parents always listened to me about not accepting those prices, and up to now I had a standard, 24/7 billing. I say up to now because it seems like somebody in the Italian system decided that everybody should be on such billing systems. I’m still on the free-market circuit (it’s still regulated, but not as strictly as the other one), but my neighbours that are on the state-regulated system and they received last month a letter stating that the energy authority will force them to move to a day/night split billing system. They suggest to run dishwashers and washing machines during the evening to save money.

Is it just me or this sounds either positively stupid or a fraud?

• Italy hasn’t been self-sufficient for energy production in a very long time, buying, as far as I can tell, the integration power from France; to the point that a fallen tree on the powerlines connecting us to France a few years ago caused a huge blackout throughout the country;
• forcing every household to have the same schedule for dishwashers and washing machines is going to put additional request the system;
• as far as I can tell, there is no reason to believe that the request of power during the day is sensibly higher than during the night; I’m actually quite sure that most of the businesses out there will keep their systems running during the night, whether they are manufacturers or third-sector offices (I have more than a couple of time-by-time customers that keep their computers running even when nobody is in office as to make sure there won’t be trouble starting up the system… I can feel why even though I don’t agree with that idea);
• if anything, power should be cheaper to provide during the day, if we were on renewable power sources like, say, sun.. I don’t think the power production during the night is higher when you’re powered by solar panels, is it?
• quite a lot of people in Italy are home throughout the day: unemployed, housewives, freelance self-employed people like me.. with tele-commuting starting to have its sense, this change seems to be against the trend.

All in all, I can feel that little part of my brain that I try to keep shut, the one that’s open to the suggestion of conspiracy theories, screaming that this sounds a lot like a way to force Italy into the kind of energy crisis that Enron shown the world how to play with. Probably it’s also because it reminds me that a bit of time ago there was quite a bit of a scandal about government officials having interest to move Italy to … coal power.

I sure hope to be found wrong here, but I have the uncanny feeling that it won’t happen.

At any rate I haven’t heard anything about similar changes for my current energy company, so I sure hope it won’t happen to me, having the tinderbox running all day, and neither me nor my mother leaving home during the day (I actually am more often out during the night, or the weekend, as I take time to go out with friends), would mean the power bill would end up quite steep.

While I would love to do as Eric does and have my computers at least to be entirely self-powered, that’s quite unfeasible to me, especially at the moment. What I would love to do would be having an easy way to turn off, say, the whole “media center” in my bedroom while I sleep or leave, or the extra elements in my office when I get out of it (such as the amplifier, or the monitor). Just having a power strip with a power switch is actually not enough, as the power connectors are in recesses that aren’t practical to reach daily.

Since there has been talking about Autotools today, and at least on the Reddit comments, my Autotools Guide got linked, I decided to take a few minutes of my time and extended the guide a bit further. I was already doing so to document the automake options (I was actually aiming at documenting the flavors and in particular the foreign mode so I would stop finding 0-sized NEWS files around), but this time I tried to make it a bit more searchable…

So right now there is a new page with the terms index and I shortened the table of concepts so that it more easily flow in the browser. The titles should be quite explaining of where to end up to. Right now I only added a single index for terms, even though I considered splitting them down per macro or variable, similarly to how it’s done in the official documentation, but for now this should do. I did add a “commons error” primary term though, as that should make it easier to find the common errors that the various tools report which I covered.

Now, these are the good news, here come the bad news though. Quite a while after first publication, the guide still is lacking a lot and my style hasn’t particularly improved. I’m not sure how good it can become by this pace. On the other hand, I’m still open to receiving requests and answering them there (thanks to Fabio asking about it, there’s now a whole section about pkg-config although it does not cover the -uninstalled variant that I use(d) on lscube so much).

Contributions in both corrections, general improvements or even just ideas are very welcome; so are donations or, more interestingly nowadays, flattr clicks (thanks to Sebastian for giving me an invite!). There is a flattr button at the bottom of the Autotools Mythbuster pages… if it is going to help you, a flattr, little as it can be, is going to show your appreciation in a way that reminds me why I started working on it.

There are going to be more news related to the guide in the future anyway, and a few more related to autotools in general, sweet news for some of you, slightly less sweet for me… so keep yourself seated, the journey is still on!

Recently, whilst travelling to Milton Keynes, 54mi north of London, for visiting HP at their newly acquired Campus (Ortensia Drive, Milton Keynes (Map), Entrance to the site), I got a room in the Hilton Milton Keynes (what a name, d’oh).

The iBahn cable in it's bag. Originating from my rooms wardrobe.

The Internet InRoom-Service at Hilton is called iBahn, what I really like, due to being such a artificial word, made up from something like Apple’s i-Products and the german word Bahn…

Wo ich wohne...

(via @janl)

I’m currently reading a book on diabetes, mostly because last year I had a bad situation with my blood sugar and this year I wanted to make sure nothing like that happens again. Winter and spring went okay, so I’m going to look at what happens this summer, as that’s always been the critical time for me. To do so, I started actually reading a book on the topic (on the Reader, obviously), partly as inspiration, partly to make sure that the indications I have gotten up to now are correct.

This all in all reminded me of something I knew already: that to actually be able to keep the blood sugar under check, I have to keep exercising. I have the Wii Fit for that (and thanks to Pavel I also have the Plus software), and I try my best to always make time for it, but indeed I have been finding excuses not to do it definitely too many times for that to be considered a proper routine. So I’m now trying to find a way to force myself keeping a decent routine.

Now I know, forcing your own body to a routine it’s not ready for is a bad idea, so I’m not trying to get myself some strict times for doing stuff. I tried that before and the result has been simply bad. But at least having an idea of what I have to do during the course of a single day and make sure I do all the points in there is something that I have to learn, especially if I intend to flee away from where I stand today.

I tried before using Remember The Milk for this stuff, but it really doesn’t work well for routines, because the way it handles repeating tasks (if you don’t complete one, it’s set as “overdue”, and when you complete it , it creates the next one right away), and because it does not support sequences, or dependencies between tasks (I cannot take my fasting blood sugar if I’ve not woken up, and I cannot have breakfast until I’ve taken my level). So I’m now looking for something new altogether. I don’t need something with a web service, with synchronisation or anything, although of course if these are available I’m not going to get angry. But it has to have a few features at least…

• it has to support sequences; as I said above each task has dependencies, since it’s a routine: wake up, check blood sugar, have breakfast, wash teeth, have lunch, check blood sugar, exercise, ….
• each entry in the sequence should hide those coming afterward until it’s done or skipped;
• each task needs to have either relative or absolute deadlines; for instance while I might wake up at different hours depending on the day, and what I did the night before, I want to wake up if possible not after 10 (but allow myself to wake up later if I was up till late, in case of exceptions) and after that I wish for my breakfast to be done before half an hour; so an absolute deadline has to be defined when actually creating the task, while the relative one is intended as “tot minutes after the previous task in the sequence completed”;
• it has to record statistics, like most videogames do nowadays; the number of days I’ve been using it, a ration between the tasks completed and those skipped and one for those completed on time or after the deadline expired, weekly, monthly and total… this will help keep me in check, as any other gamer and developer I love statistics;
• it has to be accessible as a widget so that I don’t have to open the software to know what I have to do next; even better it has to allow me to confirm the tasks’ completion or the fact that I skipped it from there as well; it also has to flash the led to warn me that some deadline expired, and possible chime in with the notification sound a few minutes before it is due, so that I can take care of the task before without it late.

Now, if you know already an application with these specifics, or something very near (better if it’s Free Software in that case), I’d like to know that… if not, but you’re good enough with Android to develop one, and then release it as Free Software, I might at least partly-fund it. I don’t have the time to develop anything like this right now and I might not even have it anytime soon, thus why I’m fine with both paying (little) for a proprietary application already present, or funding (more) the development of a Free application to do this.

As I’d to go on a business trip to London, I had the chance to compare the immigrations in times of global terrorism (as always said by the security organizations):

Immigrations @ London Heathrow on July 6th 2010

Immigrations @ Hamburg Fuhlsbuettel on July 7th 2010

So, you might have the same opinion I’ve got: a lots of queuing for nothing…

The good thing @Heathrow had been, that a border control lady opened up another lane just for us 4 (2 guys from gamigo AG and 2 guys from HP). *grin*

Das hier ist zwar keine Stellenanzeige, aber es geht um solche: Wie Unternehmen Bewerber abschrecken bei Harvard Business Manager.

Management Tip #1: Benuzt mal Eure eigene Jobs-Seite und probiert aus, wie sich das anfühlt. Tip #2: Bekommt Ihr eine Reaktion? Der Artikel zerfetzt Online-Bewerbungen und behauptet, daß einem Unternehmen mit so etwas gerade die interessante Mitarbeiter verloren gehen.

Viele der sogenannten E-Recruiting-Systeme wurden entwickelt, um die Arbeit in den Personalabteilungen leichter zu machen. Diese Motivation ist gerade bei großen Arbeitgebern, wie etwa BMW oder Daimler, nachvollziehbar. Sind diese Systeme aber auch für talentierte und motivierte Bewerber attraktiv gestaltet? Selten!

Und weil ich grad am Listen machen bin, hier eine Liste der Firefox-Plugins, die ich als Nicht-Entwickler und Geek so installiert habe.

Das E90 ist kaputt gegangen, pünktlich und zuverlässig nach 3 Jahren ließ sich der Power-Knopf nicht mehr drücken. Das neue Handy hätte auch wieder eines mit Tastatur sein sollen, aber die vom Motorola Milestone überzeugte mich nicht so, sodaß ich nun mit einem HTC Desire da stehe - das ist die HTC-Version eines Google Nexus 1.

Warum Android und kein Nokia? Wegen der User Experience, die sich nicht nur dort, sondern einheitlich bei Nokia nicht in die richtige Richtung entwickelt hat.

Was für Software habe ich auf dem Android? Basierend auf der Liste der Katze habe ich bei mir folgendes am Laufen:

I have, as a Free Software developer and enthusiast, a particular dislike for the anti-corporate websites, and the general anti-corporate feeling that seems to transpire from some of the communities that form around so-called “Free Software Advocates”. You probably know that already if you read me frequently.

In the past few days I have been again in open contrast with those trying to spread “hyperboles” which I’d sincerely call “sensationalistic name-calling”. Similarly to another point this started with one statement by Carlo Piana, who asked to stop calling “piracy” what actually is unauthorised copy. I do agree with his rational that it shouldn’t be called that way, but I’m a pragmatic, I live in this world, and like it or not, the word “piracy” as synonym for “unauthorised copy” is an unfortunate reality. Given that, you have two choice:

• keep trying to get people to use the “right term” ever and ever — the so-called GNU/Linux method;
• use their own weapons against them and (as I suggested) call piracy the disregard for copyleft licenses like the GNU GPL (note my use of words here: copyleft licenses; disregarding MIT and BSD is definitely much harder and yet they are Free Software licenses).

As I said I’m a pragmatic so it’s nothing new that I’d go with the second choice. But too many people either still think they can change the world with negative activism, or at least they pretend to, and suggested to call everything proprietary as piracy …. facepalm moment gals and guys.

I still think that most, if not all, of the people involved in anti-corporatism who pretend to care for Free Software, have no idea of what kind of effort is needed to create and maintain Free Software. Sure they might not want to be paid to do what they do, and they might have a different kind of job, so that they can do their job without “dirtying their hands” with proprietary software and proprietary vendors, but most of us, write software for a living, and usually the money come not from writing just pure Free Software — you rather have to compromise.

This does not mean that there is no business case for Free Software; we do know that a number of companies out there do Free Software mainly and can make money and pay developers to do their work, but they don’t make enough money to pay all of the people out there without at least partly compromising, leaving part of their business logic out of Free Software. Nokia and Intel, Sun before and Oracle both before and now, Canonical and RedHat, SuSE and even Apple… they all do lots of contributions to Free Software and yet their main business varies widely, just in a couple of case being mainly Free Software! Google, Yahoo and Facebook also work on Free Software, publish new, pay for maintenance of already present… yet they are not even software houses mostly (or originally).

If Free Software would require people not to be employed by companies producing any kind of proprietary software, the number of developers would be much, much reduced. Not everyone lives alone, many have a family to maintain, some have further complications, most don’t live like a hippy like Richard Stallman seems happy to. So what’s the solution? A few people, including the FSF last I checked, insist that if Free Software won’t pay for your living you can get another job, or settle for a lower wage.. but again, that is not always possible!

Do these activists put their money where their mouth is? I sincerely doubt so, as they most likely have no idea of how people sustain themselves in this environment while still keeping working on Free Software. I’ll try to give you myself as an example, but I’m sure there are situations that are more complex than mine (and quite a few that are easier, but that’s beside the point).

I don’t have to pay a rent, I’m lucky, but I’m still not working for myself alone, as I live with my mother and she’s not working. I have bills to pay each months, unhelped, that comprise phones, Internet and power, all three of which are needed for my Free Software work, as well as my “daily job” and my general living. I have obviously to buy food and general home supplies, and at the same time I have hardware to maintain, again for all the three cases. I have had a few health troubles, and I still have to both keep myself in check and be ready in case something else happens to me. I could do without entertainment expenses, but that would most likely burn myself out so I count those as an actual need as well.

In all of this, how much of the money I get is derived directly from Free Software? I’ll be honest: in the past five years, donations would probably have covered three or four months of basic need, without any saving. And mostly, that is covered by a handful of regular contributors. And before you tell me I should feel ashamed for having said this, I wish to say that I’m still very thankful to everybody who ever sent me something, be it a five euros donation, a flattr click, a book, a hardware component, or a more consistent money donation. Thank you all! Those are the things that let me keep doing what I do, as I feel it’s important for somebody.

I have written a few articles for LWN, but even that only covered a part of what I needed; the main reason is that being a non-native speaker, the time I need to write a proper article is disproportionate, again this is not to say that LWN does not pay properly – they actually pay nicely – it’s my own trouble not to be able to make a proper living from that. I actually tried finding a magazine in Italy that I could be paid to write for, getting rid of the language barrier, but the only one who ever published something (and the first article was an unpaid try) was the Italian edition of Linux Journal that has stopped publishing a couple of months later. Oh and by the way, this kind of work is also considered “proprietary work” as articles, and most books, are as far as I know not usually licensed under Creative Commons or otherwise Free licenses.

So if my pure Free Software work is not paying for bills or anything, nor my writing about it is, what am I to do? I considered for a while getting a job at the nearest Mediaworld (the Italian name for the German chain Mediamarkt), selling consumer electronics. I could do that, but then I wouldn’t probably be willing to contribute to Free Software in my spare time. What I actually do instead is, I work for companies that either make proprietary software (web software, firmware, or whatever else) or that commercialise Free Software (sorta, that’s the case for LScube for the most part). When I do, though, it often ends up with me working at least on the side for Gentoo, or Free Software in general.

I have already described my method a few months ago, I would like to say that a lot of my work on Ruby ebuilds in Portage has been done on paid time for some of my work, and the presence of gdbserver in the tree is due to a customer of mine having migrated to a Gentoo-based build system (to replace buildroot), and gdbserver was to be loaded in their firmware. A lot of the documentation I wrote also is related to that, as is my maintaining of Amazon EC2 software, …

And before this can be mistaken.. I have received more than a few job offers to do Free Software work. Most I had to turn down, either because they required me to go too much out of my way, or because of bad timing (I’m even currently in the mid of something). I also turned down Google, repeatedly, because I have no intention to ever come to USA because of my health trouble. The best offer I had was from a very well known Rails-based hosting company, I was actually very interested in the position and would have accepted even a lower wage than what I was offered, especially because Gentoo was well part of my responsibilities, but they never followed through; twice.

So anyway, what has all of this to do with the original statement, and my problem with anti-corporatism? Well, as I said most of my customers are using Free Software for developing appliances and software whose business logic is still proprietary. It’s better than nothing in the fact that they are still giving me money to keep doing what I’ve done in the past five years and counting. But at the same time, they are wary about Free Software, if they were to (like a few already do) think that Free Software is either too amateurish, or is trying to undermine their very existence entirely, they might decide that their money should not be spent on furthering those ideas.

And nothing is more dangerous than that, because if there is something that Free Software in general needs more, is competent people being paid to work mainly on Free Software. And the money often is in the hands of those companies that you’re scaring away with your “Fight da man” attitude; the same companies that Microsoft did their best to spread FUD to, regarding Linux and the Free Software and Open Source movements. I’d be surprised if there is nobody in Microsoft’s offices right now that is gloating, to see how the so-called “Advocates” are doing their best to isolate Free Software from the money it needs.

Ah yes and I was forgetting to say: if you don’t think that money is important for Free Software… take a hitch and don’t even try commenting, I will be deleting such inane and naïf comments.

Ein Photo vom Papierspender in einem WC des Flughafens von Las Vegas. Ein Glück, das der Strom für den Handtrockner aus der Steckdose kommt und keine Auswirkungen auf die Umwelt hat …

With my automated testing for Gentoo ebuilds, I started picking up quite a few ideas about automated testing. One of these is that there are three main challenges for implementing it properly: checking, analyzing, and fixing.

Checking: obviously the first obstacle you have to cope with is writing the code to check for whatever problem you’re looking out for. In the case of my tinderbox, I apply the QA tests that Portage already provides, then add some. The extended tests are almost self-explanatory, even though a few are partly Goldberg machines and thus might take a bit of insight to understand what they exactly do.

But there are quite a few parameters that you have to make sure to respect in these QA checks; one of these is that it has to have a very low signal-to-noise ratio. Removing all the noise would be pretty good, but that’s almost impossible to reach; since that’s the case, you need (as we’ll see in a moment) human work to analyse the result. But if the signal-to-noise is too high low, the test is pointless as there’s so much crap to sift through before reaching the chocolate.

It is important that the test is as much as possible non-destructive. While “destructive” in software is quite relative, I have had bad experience with QA tests that caused further failures to be injected into software, such as the “CC not respected” bugs, that expected packages to fail when the wrong path was taken. The reason why these are a bit of a problem is that when a single package breaks (forcefully), the dependency-trees its part of are also broken, so you stop another long list of packages from even being tested.

Analyzing: but even if you have tons of tests with very little SNR, you have another challenge at your hands: analysing the results, and reporting them to the right person. This is very tricky; the easiest way to cope with this is simply not to write any code to analyse the output but, instead, rely on a person to check the results and report them. This is what I’m doing; it’s “easier” but it’s very time-consuming.

The problem with analysing the errors is that sometimes they need complex interpretation; for instance while some of the GCC 4.5 errors have unique strings (the infamous constructor-specifier warning was introduced with the 4.5 version so any warning regarding that is definitely a GCC 4.5 issue), other times it can be difficult to judge whether something is a compiler bug, or an overflow that only gets uncovered by GCC itself.

It gets even worse when you add --as-needed to the mix: while it’s now mostly safe to use, the symptoms can actually appear on different projects (since shared objects often don’t use --no-undefined, it means that they can feign to have linked properly, and then leave the application linking to them to fail), or might appear in strange forms, such as an autoconf-based check reporting a library is missing while it’s actually in the system already.

But identifying the problem is just half the analysis task; the other problem is to remove the noise from the results; to do so you have to collapse duplicate problems (if one library fails because of --as-needed, all its users will fail in similar but somewhat different ways), make sure that the error applies to the latest version (especially with the way I’m running the tests here, I can receive error logs for older-than-last versions when the last fails to build — for that or for something else), make sure it wasn’t fixed in tree since I last synced (and this often I cannot do properly for time or bandwidth constraints) and also that it wasn’t reported already.

Then there is the reporting, which can also become tricky when you have test failures, as most of the ebuilds won’t use verbose testing, so you’d have to fetch the test log somehow, and since on successful merge I clear out the build directory, that’s even more tricky.

Fixing: the most difficult challenge is getting the stuff fixed, obviously. What is not obvious is what this actually mean; it’s not just matter of fixing the code but rather having someone to look at the code and intend to fix it. This basically mean that you have to report bugs that other developers will look at and not just put to the side and ignore forever.

Even if you have a test that have little to no noise at all, reporting code that is easily fixable, there is no point into spending time with the analysis for problems that will not be taken care of. They might even be important issues, but if they can’t be felt by the developers, they are not going to get solved. Portability problems come on top of that, but security is another area where we actually can’t get enough people caring. The tinderbox would be able to produce a huge number of security reports, starting from the binaries that still use unsafe functions, but most of these will not lead to any fix because “it’s working”.

So it really is important to decide what to look out for; right now, I’m trying to focus on the things that will at one point or another hit our users, unfortunately even the things that are somewhat important (Berkeley DB 5.0 or GCC 4.5 for instance) are getting ignored.

I have noticed that, beside Planet (and Universe) Gentoo and Linux-Planet — both of which I asked for my blog to be syndacated on, there are a number of other websites that seem to fetch and use my own posts, fetched from the atom feeds.

Some of these are community websites, other are “hubs” that supposedly focus content for users; quite a few though seem to be scams and websites that simply take others’ posts and snap a bunch of AdSense units over them. While I could go as far as pushing for DMCA violation for those – as often CC-BY-NC-ND is not respected – I’d rather not, and especially I’d rather go with technical solutions of some kind.

What I would like to do is to have different feeds depending on who’s requesting them; so have a “pure” feed, with the original unmangled content from Typo for the syndications I allow explicitly (with host-based detection is even better), one Flattr-enabled feed to provide to feed readers and Google, and one, eventually, with AdSense units to shove down the abusive websites.

Now, for the moment I set up the two most-subscribed feeds on Google’s FeedBurner; I’d still ask you to keep using my blog’s host URLs though, as that might not be a definitive solution; if I can find some software to deal with this, or even write a Typo plugin to handle this for me, I’ll drop FeedBurner again; I’m just not happy to have to outsource feeds handling for my own blog. I’ll call this an experiment. If you happen to subscribe to the FeedBurner feeds, don’t worry, I’ll make sure to keep them updated, and hopefully without mangling content if I do handle that on my side.

Now, the important information on this post relates to the “rules” for syndicating my content, so that it is clear whether you’ll get the Ad-encumbered feed or not. Obviously if I asked to be syndicated, as is the case for Planet Gentoo and Linux Planet, then you’ll get the absolutely clear feed: no ads, no flattr buttons, nothing else. If the website has no content, a lot of ads, and even worse no indication of where the content truly comes from, I’m going to consider technical means to hinder the syndication, such as pushing ads on the feed myself, or removing the content, or adding a footer making the author and the terms of the content’s license explicit.

For the rest of the websites, I’m generally not going to bother; if you’re a decent news aggregator, and don’t have advertisement as main content on your site, and you abide to the content’s license giving proper attribution rather than showing off my content as your own, I’ve got no problem. I’ll be happy if you were to flattr me and I might decide to push a Flattr button on the content pushed on.. if you wish to avoid that, you’re free to contact me and we can work out the details.. in general I have no objection to that as long as your aggregation software is decent enough to cache responses using If-None-Match and If-Modified-Since, and to use deflated (compressed) content transfer.

Ich bin schon länger Fan von Tims Podcasts (CRE, MM, NSFW) und möchte meinen Beitrag für die Community leisten. Die Podcasts sind seit ein paar Wochen auch via Bittorrent verfügbar und das brachte mich auf diese Idee: Mein Rootserver hat genügend Bandbreite, um einen Teil davon als Torrent-Server/Node (wie auch immer man das nennt), zur Verfügung zu stellen. Momentan läuft auf meinem Rechner zuhause µtorrent aber auf dem Server macht sich so etwas einfach besser.

Welchen Linux-Torrent-Client könnt ihr mir unter folgenden Randbedingungen empfehlen?

• keine GUI, Konsolen-Applikation
• ggf. Webinterface zum Verwalten
• RSS-tauglich um die Podcast-Feeds zu abonnieren und automatisch die Episoden herunterzuladen
• Gesamttraffic sollte begrenzbar sein

Der Client kommt dann in einen OpenVZ-Container und soll rund um die Uhr die Daten verteilen. Ich nehme auch gerne noch Tips für weitere legale Torrents entgegen, die Unterstützung gebrauchen könnten.

Eure Vorschläge? Danke!

RFC: Bittorrent auf dem Rootserver is a post from: nodomain.cc

Related posts:

1. Howto: Rootserver als OpenVPN-Gateway nutzen
2. Lifestream-Gefrickel
3. 30gigs.com - Squirrelmail auf Rootserver?

I have to thank Arfrerver for making me notice this with the bug about Ruby 1.9 he reported.

The GNU project released autoconf 2.66 two days ago. Very few notable changes are present in it, just like a few were listed before, so I didn’t go out of my way to test it beforehand. My bad! Indeed there is one big nasty change with it for which I’d say to all of you to put off the update until I write it so. Hopefully it won’t get unmasked in Gentoo for a while either.

There are two main problems with this release; the first is due to the implementation of a stricter macro to ensure the parameters given to it is not variable over executions:

• The macro AS_LITERAL_IF is slightly more conservative; text containing shell quotes are no longer treated as literals. Furthermore, a new macro, AS_LITERAL_WORD_IF, adds an additional level of checking that no whitespace occurs in literals.
  

well, whatever the idea about this was, it seems to have broken the AC_CHECK_SIZEOF macro: if you pass it [void*] as parameter, it’ll report it not being a literal (while it is) causing the following error:

flame@yamato test % cat configure.ac
AC_INIT([foo], [0])

AC_CHECK_SIZEOF([void*])

AC_OUTPUT

flame@yamato test % autoconf
configure.ac:3: error: AC_CHECK_SIZEOF: requires literal arguments
../../lib/autoconf/types.m4:765: AC_CHECK_SIZEOF is expanded from...
configure.ac:3: the top level
autom4te-2.66: /usr/bin/m4 failed with exit status: 1

This would be bad enough. But the nastier surprise I got when running autoreconf over the feng sources, the build system of which I wrote myself, and if I may say so, is very well engineered:

flame@yamato feng % autoreconf -fis
configure:6275: error: possibly undefined macro: AS_MESSAGE_LOG_FDdnl
      If this token and others are legitimate, please use m4_pattern_allow.
      See the Autoconf documentation.
autoreconf-2.66: /usr/bin/autoconf-2.66 failed with exit status: 1

The problem here is almost obvious, and it’s related to the dnl entry at end of the macro name; the dnl keyword is used as (advanced) comment delimiter in autoconf scripts, meaning “Discard up to New Line” and is often used to keep on multiple lines commands that should be kept togever, like \ is in many languages. A quick check at the configure files brings in this:

        as_fn_error $? "Package requirements (glib-2.0 >= 2.16 gthread-2.0) were not met:

$GLIB_PKG_ERRORS

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables GLIB_CFLAGS
and GLIB_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details." "$LINENO" AS_MESSAGE_LOG_FDdnl

You can easily see that the problem here is with the pkg-config macros (pkg.m4). Funnily enough there is no change related to the errors reporting that is listed in the autoconf news file so I wasn’t expecting this. The problem is further down the path of pkg-config files but it’s not important to fully debug it right now, it’s actually quite easy to fix, in pkg-config itself, but here’s the catch.

Since the pkg.m4 macro file is way too often bundled with the upstream packaging, and its presence overrides the copy from the system, even fixing pkg-config will not fix all the software that carries outdated copies of the macro file.

This is almost the same problem with libtool 1 vs libtool 2 macro files with the difference that this is going to be much much more common. If you’re a package maintainer, you can do something already before this even hits the users: remove the pkg.m4 file during the src_prepare() phase; you’re already depending on pkg-config in the ebuild for it to work at build-time, and since we don’t split the macro file from the command itself, you can simply rely on its presence on the system.

In the mean time, I’m not sure if I want to start testing with it just yet or if we should be waiting for 2.67…

I know that most of you who read my blog daily don’t care about my toying with eBooks, and only read it for the technical articles; on the other hand, I feel like I can at least talk a bit about that, given that most of my personal life is uninteresting and thus I rarely write of that at all.

Anyway, you might remember I had some trouble finding where to buy eBooks and at the end I settled with – for non-technical books that is – WHSmith and Kobo as they both sell Adobe Digital Edition ePub books. Finding mainstream non-DRM ePub seems to be impossible; maybe only on Apple’s iBooks store, but it still doesn’t warrant me getting an iPad to try — even though, if you have an iPad or iPhone and can tell me whether that’s the case, I’d be curious. Finding a second-hand old-generation iPhone shouldn’t be too expensive and if that can get me access to mainstream non-DRM’d ePubs it might be worth it.

Anyway, the two sites above actually give me enough access that I don’t miss most of what I usually read; indeed, Kobo actually provided me with a few curious readings that I might as well try. Also, even though the Dollar is rising again, buying the books from Kobo is, for me, slightly cheaper than WHSmith.

Also, the fact that they are no simple eBook store makes them more intriguing; I’m not that enticed by their eReader (given I have already my PRS-505 and I’m not going to drop it any time soon), but the fact that they have applications available for a number of platforms (but not Linux, dang it! If they did, and it supported activation of Adobe DRM’d ePubs, they would be so great I could consider getting the eReader if only to fund them further). Even if I will probably not use those, I can still enjoy the fact that they let me read the books I buy on the web with any browser, on my reader in ePub format (and thus anywhere the ePub format can be read!) and since a few days also on my Milestone thanks to their Android application.

A word about the DRM here; while I’m one of those people who, I said already, prefer to abide to restrictions as long as they are an acceptable tradeoff (for instance the audiobooks DRM on iTunes is acceptable because they do cost a lot less than on unencumbered form). While I can understand the reason why most publishers won’t even consider not using DRM on the files, and I accept that at least this way I can get eBooks at all, I don’t think the tradeoff is useful to the user in this case. Indeed, given the fact that not all devices using ePub supports Adobe Digital Editions, it can be quite harsh to have it applied. add that to the not all ePubs are the same and thus you might have to access the content of the archive to change it into something usable, and you get the picture. Luckily, the ADEPT DRM has been long broken so it’s not difficult to get clean files.

Anyway, as I said, Kobo looks a nice choice to me because of the presence of the additional applications (just to put it into perspective, while I’m not considering buying an iPad, were I to, I could still read the books I bought from Kobo, without going around the DRM, as they have an iPad application); for instance I could easily read The Salmon of Doubt from my browser, even though the ePub version uses the infamous DTBook format above. Unfortunately they don’t have everything… not yet at least.

Anyway, last night I didn’t sleep so I could finish reading Assassin’s Apprentice (somebody suggested this to me a few years back; on the other hand I decided to read this because me and some friends were to a fair where also the author was…). Nice book indeed, just a bit “slow” (took me almost a month to read it fully, and it was just 400 pages). Next step, though, I wanted to come back to Dresden’s Files; Butcher’s style is enchanting. Three books out, I was up to read Summer Knight — Grave Peril I got from Kobo so I assumed they had the next as well; somehow, they don’t. So at the end I got it from WHSmith; it bears little difference, but it still strikes me as odd.

And in all this, there seems to be no shop for Italian eBooks; sigh. If only ChiareLettere had ePubs available.. their books are quite bulky and I would love to give them away and trading them for digital copies of them. I wonder if I should get more (technical) skills about this kind of publishing and propose to handle that kind of stuff myself. I would also know where to start, maybe.

Ich habe mich nun lange genug zurückgehalten. Gestern ist es passiert, ich habe die Originalfirmware auf meinem HTC Desire durch Android 2.2 ersetzt. Ich hatte einfach keine Lust mehr, darauf zu warten bis HTC die offizielle Firmware veröffentlicht und ich auch in den Genuss der tollen neuen Features kommen kann.

Natürlich musste ich das Gerät dazu erst "rooten". Das geht eigentlich ganz einfach.

Rooten

Rooten bezeichnet den Vorgang, bei dem man sich die Rechte auf dem Gerät verschafft, die einem als zahlender Kunde eigentlich zustehen . Dazu benötigt man im Falle des HTC Desire eine sog. Goldcard. Das ist eine SD-Karte mit speziellen Modifikationen. Ich bin dabei nach dieser Anleitung vorgegangen.

Sobald man die Goldcard im Handy hat, kann man mit dem Rooten beginnen. Dabei habe ich mich an diese Anleitung gehalten und es hat auf Anhieb funktioniert. Danach hat man ein "Stock ROM", d.h. eine ungebrandete Version des originalen HTC Desire ROMs.

ROM Manager

Um es sich möglichst einfach zu machen, installiert man nun den ROM Manager aus dem Market. Sobald der ROM Manager installiert ist, wählt man dort das ClockworkMod Recovery zur Installation aus. Dies ist ein spezielles Recovery-Image, mit dem man die Custom ROMs flashen kann. Außerdem kann man Backups (nandroid) erstellen und wieder einspielen, die SD-Karte partitionieren usw.

Ich hatte gestern das Problem, dass die Website androidaftermarket.com, auf die der ROM Manager zum Download aller ROMs zugreifen muss, down war. Deswegen habe ich eine andere Möglichkeit gefunden, das Recovery Image auszutauschen. Danach war eine Installation von DeFroST kein Problem mehr. Dazu muss man nur noch das heruntergeladene Update.zip auf die SD-Karte kopieren und im Recovery das Image flashen. Vorher den "Wipe" nicht vergessen!

Den Recovery-Modus erreicht man durch Drücken von Power+Back bei ausgeschaltetem Desire. Im nun erscheinenden Menü per Volume Up/Down zu "Recovery" navigieren und mit "Power" bestätigen.

DeFroST

Ich habe mich für das DeFroST ROM entschieden, da ich von @mthie den Tipp bekommen hatte. Es läuft wirklich stabil und zeigt deutlich, dass Android 2.2 im Vergleich zu 2.1 einen weiteren Performancevorsprung bietet.

Heute gab es bereits wieder ein Update, was ich ohne zusätzlichen PC direkt "OTA" über den ROM Manager installieren konnte.

Mir gefällt das "rohe" Android ohne Sense wirklich besser. Das einzige was mir nun fehlen wird, ist die Outlook-Synchronisation für meine geschäftlichen Termine - aber dafür werde ich mir Companionlink nochmal anschauen.

Fazit

Während des Flashens spielt der Puls natürlich verrückt aber insgesamt verlief die Aktion echt problemlos. Ich bin froh, dass ich mich endlich getraut habe ... Danke nochmals an @mthie für den letzten Motivationsschub!

Wer wissen möchte, wie man auf einem Nexus One von Vodafone bereits jetzt Android 2.2 installieren kann, der sollte bei Simon vorbeiklicken.

Android 2.2 "Froyo" auf dem Desire is a post from: nodomain.cc

Related posts:

1. Mein neues HTC Desire
2. Android 2.2. mit Flash

Dies ist der letzte Tag der Networkers. Ich habe mir die Sessions “Deploying Performance Routing”, “Network Diagnosis: Prevent Prepare Repair” und “Troubleshooting GET VPN Deployments” herausgesucht, die alle recht gut waren.
Die letzte Keynote mit Ben Mezrich habe ich dann auch ausgelassen, da das alles zu hektisch gewesen wäre. Die Meinungen der Keynote-Besucher waren teilweise auch sehr durchwachsen. Wenn allerdings noch einmal ein Sprecher wie z.B. John Cleese kommen würde, dann wäre natürlich klar das man die wieder mitnehmen würde.

Damit ist dann diese Networkers-Woche vorbei. Wer im nächsten Jahr auch die CiscoLive! besuchen möchte, sie ist wieder im Mandalay Bay Convention Center, vom 10. -14. Juli. Der Termin ist bei mir schon im Kalender eingetragen.

I’ve now redirected the tinderbox to run the build with OpenSSL 1.0.0a, to assess how much of the tree still needs to be ported before moving to the new OpenSSL release; this is another tricky update if you don’t use --as-needed, so, start using it now.

Similarly to what happened with GCC 4.5 and the combined libpng/berkdb testing I had to clear up the whole tinderbox, and start from scratch after rebuilding the system. But unlike those cases, I didn’t have to rebuild everything in the base system of the tinderbox (in the case of GCC it is needed; in the case of the combined update it was simply faster), so I had the time to look a bit more thoroughly to what happened.

Do you remember my woes with SELinux and libxcrypt ? I had a similar surprise, but at least this time it didn’t kick me out of the tinderbox. Again, automagic dependencies, and not just in the base system of the tinderbox but also for packages directly into the system set. And a bunch that are present indirectly.

But first, what on Gentoo is an automagic dependency? Well, an automagic dependency can be either hard or soft… a hard-automagic dependency is a dependency that the upstream build system detect the present of automatically and provides no way to override it; these are the worst situations because it means that you have to hack at the upstream build system to provide an override; it’s for this kind of situations that I started the Automagic fixing guide and it’s one of the first thing that I documented in Autotools Mythbuster so that it can be fixed as intended.

A soft-automagic dependency is one that behaves in the same way but for which upstream actually provided a switch, but the ebuild does not grok and does not export as USE flag properly. This happens because either the ebuild simply lacks the code out of mistake or because the author of the ebuild thought something along the lines of “oh we don’t have foo in Portage, so I cannot set an USE flag, I’ll just leave it as it is”. It is a mistake, a big one, always disable optional features depending on software we don’t have in Portage so that if that software is ever introduced you don’t get yourself an automagic dependency. I’ll show you in a moment that this is exactly what happened.

First problem first, both Python version decided to fail the rebuild with OpenSSL 1 while I did know they should have worked, a quick glance noted that the problem was indirect: dev-lang/tk has an automagic dependency over libXss (the implementation of the XScreenSaver protocol); this obviously made it fail to build the tkinker extension that is needed by so many packages. D’oh! There is also another automagic dependency over a different library, directly in Python, but I hadn’t tracked it down further yet.

I was also able to identify an automagic dependency in qt-sql and another one (unrelated to this cleanup) in qemu; both have been reported together with the ones above.

But what concerned me was an automagic dependency in GNU gettext. This is nasty because this package is part of the system set, and since the system set is what the stages are comprised of, this makes it trouble. While in most cases automagic dependencies are mildly annoying, an exception being the Linux-PAM problem I noted above, and another is system set packages, for the way they are built, by using ROOT= make them vulnerable to automagic dependencies on the host system.

In the case of gettext, the problem was the recently-introduced GNU libunistring, needed for guile and used by gettext 0.1.8.1 and later, if present. It also became quite a trick to properly disable its usage: the macros used to check it out and enable it are taken by that monstrous mess that is called gnulib, and they don’t respond correctly to --without-libunistring-prefix as ./configure would have suggested. On the other hand, --with-included-libunistring worked as a charm, without causing bundled libraries problems because there is no bundled libunistring. And now try telling me again that GNU developers know how to use autotools.

For now, the most important one is covered, without revbump if you’re a guile user and have libunistring installed, you might want to rebuild gettext after a sync, just to be on the safe side.

But learn the lesson, and always --disable and --without the optional dependencies that your package would use if they were present. Do it for you, me and the rest of the users. Pretty please, with sugar!

Meine Sessions des Tages waren “Cisco Integrated Services Routers G2 – Architectural Overview and Use Cases”, “Deploying and Troubleshooting Web Cache Communication Protocol (WCCP)” und “Enterprise IPv6 Deployment”.
Die IPv6-Session habe ich auf einer der letzte Networkers schon einmal angehört, aber da sich in diesem Bereich doch noch einiges in der Entwicklung befindet, kann man diese Session ruhig ab und an wiederholen. Interessant war, auf welches Interesse IPv6 inzwischen stößt. Es waren 396 Teilnehmer zu dieser Session registriert, ca. 3/4 davon waren anwesend. Das war bisher die Session mit den meisten Teilnehmern, in der ich war. In den letzten Jahren waren das durch die Bank weg weniger Interessierte.

Abends war dann der Customer Appreciation Event, der in der Garden Arena des MGM Grand stattgefunden hat. Das ist die Arena, in der dieses Jahr Lady Gaga auftritt, oder aber vor einigen Jahren Mike Tyson sich das Ohr von Evander Holyfield hat schmecken lassen. Mehrere Bands sorgten für Stimmung, für Essen war auch gesorgt und die Getränke flossen auch mal wieder in Strömen. Eine Kamera hatte ich leider nicht mit, daher kann ich den diesjährigen Cisco-Hut (noch) nicht zeigen … Aber er passte wieder zu “Viva Las Vegas!”. Dank der Handy-Kamera meines Trainer-Kollegen Helge, hier eine Variation (es gab den “Hut” in diversen Farben):

Der Hut zur CiscoLive! 2010

Heute standen mehrere Punkte auf der Tagesordnung:

1) Rezertifizierung.
Eine Vue-Prüfung ist bei der Cisco Networkers immer kostenlos. Das ist eine sehr gute Gelegenheit, die relativ teure CCIE-Written-Prüfung zur Rezertifizierung zu machen. Diesmal habe ich wieder die CCIE-Security-Prüfung gemacht, und bin jetzt bis 2013 rezertifiziert.

2) Die Keynote mit John Chambers.

Die Visionen, die John Chambers versucht zu kommunizieren, sind — wie eigentlich jedes Jahr — mal wieder nicht bei mir angekommen. Aber interessant war ein Gerät, das er vorgestellt hat. Ein Tablet-Computer, der nicht nur ein mehr oder weniger offenes System hat (Android), sondern auch Multitasking, USB, einen SD-Card-Slot und AGN-WLAN. Natürlich sind das alles Sachen, die ich als iPad-User als sinnlos empfinde; zumindest solange, bis das iPad so etwas auch hat. Aber was einen neidisch machen kann ist, das dieses Cisco CIUS Telepresence kann. Und das könnte nicht mal ein echter Apple Fan-Boy (als den ich mich aber eigentlich nicht sehe) abstreiten, dass das einfach klasse ist.
Weiterhin wurde gesagt, das dieses Jahr ca. 12500 Teilnehmer zur CiscoLive! in Las Vegas sind.

3) Meine Sessions des Tages waren dann “Server Load Balancing Design” und “IPv4 Exhaustion: NAT and Transition to IPv6″. Während die erste Session enttäuschend war, da sie kein Design beinhaltete, sondern nur grundsätzliche Konfigurationen, war die zweite sehr interessant. Die Mechanismen, und vor allem die Probleme vom NAPT444, NAT46, NAT64 und NAT66 wurden beleuchtet. Zusätzlich gab es gute Informationen zu IPv6rd.
Zwischen den Sessions war die “Cisco Live Network Operations Center Tour”. Es wurde gezeigt, wie das Konferenz-Netzwerk funktioniert. Auch wenn es im WLAN ein paar Aussetzer gab, ist es faszinierend, wie das Team in kurzer Zeit ein WLAN mit 150 Access-Points aufgebaut hat.

4) Anschließend war das CCIE-NetVet-Treffen mit John Chambers. Da sich das mit der letzten Session überschnitt (die gleiche sonderbare Planung wie letztes Jahr), habe ich zwar nur die Hälfte mitbekommen, aber ein paar Informationen waren sehr interessant. Es klang zwischen den Zeilen heraus, als wenn nicht nur der CSA stirbt, sondern auch die Cisco MARS das gleiche Schicksal erleiden könnte.

5) Die CCIE-Party. Das war der Höhepunkt des Tages. Die Voodoo-Lounge auf dem Dach des Rio-Hotels (51. Stock) war für uns reserviert und es gab allerfeinstes Essen und dazu Getränke wie den “Witch Doctor”. Die Aussicht von dort oben ist einfach bombastisch:

Zum Abschied nach der Party gab es natürlich noch ein paar Geschenke. Neben einer CCIE-Voodoo-Lounge-Tasche gab es ein Kartenspiel, Würfel, und einen Kofferanhänger, alles mit dem CCIE-Logo. Dazu noch einen Kühlschrank-Magneten mit dem Rio:

Und dann ist mir vormittags noch aufgefallen, mit welcher Software Cisco ihre Kiosk-PCs schützt, und wie gut das geklappt hat …

Some people thought that my previous blog about the libpng debacle was meant as an attack, or a derision, of the work and effort that Samuli put into getting us out of the libpng-1.2 mess we were. Let me be clear: it wasn’t. I’m glad that Samuli is there, without him I would probably have left Gentoo a long time ago, frustrated by nothing happening.

But I think that we shouldn’t hide our head under the sand and keep repeating “it’s all good, it’s all good”. It isn’t.

Samuli did the best he could to get us out of the trouble, which is much bigger than a single person, two, three or even a dozen could properly tackle with all the possible bases covered, at this point, unless there is consensus among the whole developer body, which isn’t there.

But first, I have to say that one thing I’m going to maintain was done wrong, in the rush of the moment: stabling libpng-1.4 as part of a security fix. That was simply reckless. But the fault does not lie in a single person, but rather in the general spirit of avoiding doing extra work… still, reckless or not, it’s done and we have to live with it, and learn from it.

And learning seems like we are; finally there is enough traction for --as-needed to become default as I wrote recently and I got to thank Samuli and Kacper without whom we wouldn’t be able to reach that point at all. I unfortunately still don’t see the same traction behind the hidea of dropping .la files. Removing them from gtk+ which doesn’t install any static library and thus does not need the .la files at all, would have solved if not all, most of the problems people had with the upgrade…

Oh and by the way, the update script for libpng is a hack and it will leave behind .la files when packages will start dropping them, as it changes their checksum and timestamp without updating the package database. The same is true for the (generic) lafilefixer which is why I’ll recommend again to apply the incremental one, as declared in the two posts linked at the beginning.

Finally, libpng-1.5 is going to be released sometime soon… either we make a plan now, or we’re going to suffer through another identical pain soon. And libpng is known for having security issues quite often… I already sent Samuli the plan I was thinking on this morning; I’ll write more details about that as I find the time.

Dirk von Gehlen schreibt in Journalistisches Freibier über den Kampfbegriff der 'Kostenloskultur' im Internet und daß es diese so nicht gebe. Er sagt

Es ist dringend notwendig, neue Erlösmodelle für verlegerische Angebote im Netz zu entwickeln. Vermutlich wird das aber so lange nicht gelingen, wie dieses Missverständnis der Umsonst-Kultur im Raum steht.

...

Mir ist niemand bekannt, der an diese Umsonst-Kultur glauben würde. Die gibt es nämlich nicht. Es ist vielmehr eine Kultur, die auf anderen Finanzierungsmodellen beruht als die klassische Bezahlkultur.

...

Ähnlich wie die falsche Rede vom Diebstahl zeigt die vermeintliche Kostenlos-Kultur, dass es an den sprachlichen Mitteln fehlt, die Veränderungen der Digitalisierung zu fassen (...). Vielleicht ist es tatsächlich so, dass wir die Revolution, die das Internet angestoßen hat, erst dann produktiv nutzen können, wenn wir Begriffe gefunden haben für das, was sich gerade verändert.

A few months ago I criticised Qt team for not following QA indications regarding the installation of documentation. Now, I wish to apologize and thank them: they were tremendously useful in identifying one area of Gentoo that needs fixes in both the QA policy and the actual use of it.

The problem: the default QA rules, and the policies encoded in both the Ebuild HOWTO (that should be deprecated!) and Portage itself, is to install the documentation of packages into /usr/share/doc/${PF}, a path that changes between different revisions of the same package. Some packages currently don’t respect that; some because it wasn’t thought about; some because they were mistakenly bound to ${P} on a zero-revision ebuild; some because they need not respect that paths.

When I started reporting for wrongly-installed documentation, I wasn’t expecting any in the latter category, it turns out to find so many examples of the latter; and a further number of examples and use cases that would call to change that policy altogether:

• package foo requires to know where package bar installs its documentation, so that it can load it up, for whatever reason that is; this requires bar to either symlink its documentation somewhere or break the current policy, or otherwise you’d have to rebuild foo each time to find the correct path, which is not feasible;
• package baz requires to know where its own documentation is installed to be able to access it at runtime; this either requires it to be hardcoded in the sources or to write it in a configuration file requiring semi-manual merge through etc-config; this is the case of Postfix for instance;
• probably most important for users, API documentation bookmarks, right now, cannot be made stale unless you use symlinks; this is very annoying for the people who use those packages to develop (and you might guess that my main target here would be Ruby gems).

The solution: not sure if I can say I have a solution, but Samuli and Ulrich proposed a number of possible alternatives to solve the problem; from their suggestions I’d say we have to encode exactly three informations. The category of the package, the package name, and the slot of the package itself — the category is needed because there are a number of packages with the same name and different categories… sometimes even with the same version (dev-php5 and the old dev-php4 categories are a good example of those, and they were systematically breaking the policy stated above).

One solution I was proposed was /usr/share/doc/${CATEGORY}_${PN}-${SLOT} which wouldn’t be bad… but it would have a -0 appended to most of the directories; my preferred solution there would be to do something like omitting -${SLOT} if it’s 0. You’d have stable API documentation links, most of the intra-package and inter-package paths would be stable, and all in all you could drop the need for the document symlinking feature we currently have.

Unfortunately I’m expecting this to either require an EAPI bump or it’ll take a number of years before this can be properly implemented; I’ll probably have to either author myself – or find someone to author it for me – a GLEP to suggest changing dodoc. Contextually we should consider finding a better solution for compression, which is another problem we hit. Right now only the documentation installed with dodoc is getting compressed with the chosen compression program, which might be gzip, bzip2 or lzma, same as the man pages. While man pages gets processed after install and before binpkg/livefs merge steps, documentation is not.

But not all documentation needs to be compressed in the first place: HTML files (API documentation first of all); PDF files; code examples need to be accessible without compression; while we have a dohtml command to installing the web pages without compressing them, there is no equivalent for the other, and we have to rely on insinto/doins pairs. Further on, with more and more autotools-based packages moving to autoconf 2.6x and supporting the --docdir option, we’re going to install more and more documentation directly into the directory, be it with the current ${PF} or other form; these won’t be compresses as they are, right now.

So, again thanks to Ben for actually challenging the status-quo; his insights here were the spark that made me think about this for a long time.

Seems like my previous post didn’t make enough of a fuss to get other developers to work on feasible solutions to avoid the problem to hit stable users… and now we’re back to square one for stable users.

Since I also stumbled across two problems today while updating my stable chroots and containers, that represent the local install of remote vservers, and a couple of testing environments for my work, I guess it’s worth writing of a couple of tricks you might want to know before proceeding.

Supposedly, you should be able to properly complete the update without running the libpng-1.4.x-update.sh hack! (and this is important because that hack will create a number of problems on the longer run, so please try to avoid it!). If you have been using --as-needed for a decent amount of time, the update should come almost painless. Almost.

I maintained that revdep-rebuild should be enough to take care of the update for you, but it comes with a few tricks here that make it slightly more complex. First of all, the libpng-1.4 package will try to “preserve” the old library by copying it inside itself, avoiding dynamic link breakage. This supposedly makes for a better user experience as you won’t hit packages that fail to start up for missing libraries, but has two effects; one is that you may be running a program with both libpng objects around, which is not safe; the second is that revdep-rebuild will not pick up the broken binaries at all, this way.

Additionally, there is a slot of the package that will bring in only the library itself, so that binary-only packages linked to the old libpng can be used still; if you have packages such as Opera installed, you might have this package brought in on your system; this will further complicate matters because it will then collide with libpng-1.4… bad thing.

These are my suggested instructions:

• get a console login, make sure that GNOME, KDE, any other graphical interface is not running; this is particularly important because you might otherwise experience applications that crash mid-runtime;
• emerge -C =libpng-1.2* make sure that you don’t have the old library around; this works for both the old complete package and for the new library-only binary compatibility package;
• rm -f /usr/lib/libpng12.so* (replace lib/ with lib64/ on x86-64 hosts; this way you won’t have the old libraries around at all; actually this should be a no-op since you removed it, but this way you ensure you don’t have them around if you had already updated;
• emerge -1 =libpng-1.4* installs libpng-1.4 without preserving the libraries above; if you had already updated, please do this anyway, this way you’ll make sure it registers the lack of the preserved libraries;
• revdep-rebuild -- --keep-going it shouldn’t stop anywhere now, but since it might, it’s still a good idea to let it build as much as it can.

Make also sure you follow my suggestion of running lafilefixer incrementally after every merge, that way you won’t risk too much breakage when the .la files gets dropped (which I hope we’ll start doing systematically soon), by adding this snipped to your /etc/portage/bashrc:

post_src_install() {
    lafilefixer "${D}"
}

Important, if you’re using binary packages! Make sure to re-build =libpng-1.4* after you deleted the file, if you had updated it before; otherwise the package will have preserved the files, and will pack it up in the tbz2 file, reinstalling it every time you merged the binary.

This post brought to you by the guy who has been working the past four years to make sure that this problem is reduced to manageable size, and that has been attacked, defamed, insulted and so on so forth for expecting other developers to spend more time testing their packages. If you find this useful, you might want to consider thanking him somehow …

Heute standen drei Sessions auf dem Programm:

1) Next-Generation Network Access Policy with Cisco ACS
Hier war die Arbeitsweise des ACS 5.x das Thema. Bei meinen Kunden habe ich bisher nur die 4.x-Versionen eingesetzt. Die neue Version wird komplett anders konfiguriert und verwaltet. Die Flexibilität macht aber klar, dass diese ACS-Version bei der nächsten Implementierung eingesetzt wird.
Nachdem gestern beim Techtorial alle gefroren haben (gefühlt ca. 10 Grad), gab es bei dieser Session das andere Extrem. Der Raum war völlig überhitzt und alle, die vom Vortag gelernt haben und in langer Hose und langem Hemd gekommen sind, haben geschwitzt. Ist halt schwer …

2) Incorporating Intelligent Access at the Campus Edge
Neue und erweiterte Protokolle bzw. Funktionen für die Userports wurden vorgestellt. POE+, das bis 30 Watt liefert, LLDP vs. CDP, EnergyWise, Neuerungen beim AutoQos sowie neue Smartport-Makros, die jetzt evtl. sogar benutzbar werden. Am Ende wurde “Smart Install” vorgestellt, mit der neue Switche automatisch im Netz mit der richtigen IOS-Version und Konfig bestückt werden können.

3) LISP – A Next Generation Networking Architecture
Das war die beste Session bisher. LISP bringt Erweiterungen, die hauptsächlich für multihomed-customers sinnvoll sind. Dabei wird die Identity (die IP des Endgerätes) von der Location (dem Anschluss an das ISP-Netz) getrennt. Eine extrem spannende Technologie, die aber noch nicht für den produktiven Einsatz verfügbar ist.

Und dann musste man aufpassen mit welchem WLAN man sich verbindet. Das ein oder andere würde das Budget doch etwas sprengen:

Abends war heute natürlich auch die “Welcome Reception” zur Eröffnung der World of Solutions. Das ist dann auch gleichzeitig der Tag, an dem einem von den Ausstellern die kostenlosen T-Shirts nur so aufgedrängt werden; ob man will, oder nicht.

Global Knowledge und Netapp

Splunk mit “Log, I’m your Father” und New Horizons

Das Solarwinds-Shirt ist nicht getragen, sondern war zu einer kleinen Flamme zusammengepresst; Fluke und Compuware

Das Cisco-Shirt gab es für die Teilnahme am Videoblog, von WhatsUpGold gab es eine Wasserflasche. Das Beste ist aber der “Pointy-haired Boss (PHB)”, der auf den Schreibtisch kommt. Ich weiß jetzt aber nicht mehr, welche Firma den verteilt hat …

Tag 0: Die Networkers ist gestartet. Gestern war nur Registrierung und man konnte seine Unterlagen abholen. Der Rucksack (unten auf dem Bild) macht einen qualitativ recht hochwertigen Eindruck. Auch wird wieder auf Einweg-Wasserflaschen verzichtet, man kann sich Wasser aus Spendern in die zugehörige Flasche abfüllen. Die Session-Unterlagen gab es dieses Mal auch wieder auf einem USB-Stick (ich wusste aber nicht, dass es überhaupt noch 2GB-Sticks gibt ), nachdem sie letztes Jahr nur online verfügbar waren. Das ist ganz praktisch, da kann man auf dem iPad bei Bedarf zurückblättern, wenn es vorne zu schnell geht.

Tag 1: Heute waren Techtorials. Ich habe mir das Thema “Unleashing the ASA” herausgesucht. Der Anfang war recht gut und einige Themen wurden behandelt, mit denen ich mich noch nicht beschäftigt habe. Leider ist die Session später etwas schlechter geworden, da wieder zu viele Basics behandelt wurden, die für eine “unleash”-Session eigentlich Voraussetzungen sein sollten. Egal, ein paar Ideen habe ich trotzdem mitgenommen.

Die WLAN-Versorgung war auch dieses Jahr wie immer am ersten Tag der Networkers. Sie funktionierte zumindest vormittags einfach nicht. Nachmittags wurde es dann etwas besser. Aber auch das kennt man ja schon.

Der CiscoLive 2010 Rucksack

I’ve been having some off time (well, mostly time I needed to do something that kept me away from facebooker craziness since that was driving me crazy quite literally), and I decided to get more work to do in Ruby-Elf (which now has its own page on my site — and a Flattr button as well, like this blog and Autotools Mythbuster thanks to Sebastian.

What I’m working on right now is supporting C++ name demangling, in pure Ruby; the reason for that is that I wanted to try something “easier” before moving on to trying to implement the full DWARF specification in Ruby-Elf. And my reason to wishing for a DWARF parser in there is that Måns confirmed my suspicion that it is possible to statically-analyze the size of the stack used by a function (well, with some limitations, but let’s not dig into that right now). At any rate I decided to take a stab at that because it would come useful for the other tools in Ruby-Elf.

Now, while I could assume that most of my readers already know what a demangler is (and thus what is mangling), I’ll see to introduce it for all the others who would otherwise end up bored by my writing. C++ provides a much harder challenge for linkers and loaders, because the symbols are no longer identified just by their “simple” name; C++ symbols have to encode namespaces and class levels, a number of special functions (the operators), and in the case of functions and operators, the list of parameters (because two functions with the same name and different set of parameters, in C++, are valid). To do so, all compilers implement some kind of scheme to translate the symbols into identifiers that use a very limited subset of ASCII characters.

Different compilers use different schemes to do so; sometimes even differing depending on the operating system they are building on (mostly that’s for compatibility with other “native” compilers on that platform). The scheme that you can commonly find on Linux binaries is the so-called GCC3-mangling that is used, as the name leaves to intend, by GCC 3 and later and by ICC on Linux (and iirc OSX). The complete definition of the mangling scheme is available as part of the Itanium ABI definition (thanks to Luca who found me the link to that document); you can recognize the symbols mangled under this algorithm by their starting with _Z. Other mangling schemes are described among other documentation — link provided by Dark Shikari.

While standardising over the same mangling algorithm has been proposed many times, compilers make use of the difference in mangling scheme to prevent risk of cross-ABI linking. And don’t expect that the compilers will standardise their ABI anytime soon.

At any rate, the algorithm itself looks easy at the first glance; most of the names are encoded by having the length of the name encoded in ASCII and then the name itself; so an object bar in a namespace foo would be encoded as _ZN3foo3barE (N-E delimit the global object name). Unfortunately when you add more details in, the complexity increases, a lot. To avoid repeating the namespace specification time after time, when dealing with objects in the same namespace, or repeating the full type name when accepting objects of its own class, or multiple parameters with the same type, for instance, the algorithm supports “backreferences” (registers); name fragments and typenames (but not function names) are saved into these registers and then recovered with specific character sequences.

Luckily, Ragel helps a lot to parse this kind of specifications; unfortunately I have reached a point where proceeding further require definitely a lot more work than I’d have expected. The problem is, you have recursion within the state machines: a parameter list might contain.. another parameter list (for function pointers); a type might contain a typelist, as part of a template… and doing this in Ragel is far from easy.

There are also shorthands used to replace common name fragments, such as std:: or std::allocator that would be used so many times that the symbol names would be exceedingly huge. All in all, it’s a quite complex operation. And not a perfect one either; different symbols can demangle to the same string; and not even the c++filt shipping with binutils take care of validating the symbol names it finds, so for instance _Zdl is translated to “operator delete” even though there is nothing like that in C++: you need a parameter for that operator to work as intended. I wonder if this can be used to obscure proprietary libraries interfaces…

Now, since Luca also asked about this, I’d have to add, as Måns already confirmed, that having too-long symbol names can slow down the startup of a program; in particular it increases the time needed for bindings; even though generally-speaking the loader will not compare the symbol name itself, but rather its hash on the hash table, the longer the symbol the more work the hash function has to do. And to give an idea of how long a name we’re talking about, take for instance the following real symbol; exported symbol, nonetheless, coming from gnash:

_ZN5gnash13iterator_findERN5boost11multi_index21multi_index_containerINS_8PropertyENS1_10indexed_byINS1_14ordered_uniqueINS1_13const_mem_funIS3_RKNS_9ObjectURIEXadL_ZNKS3_3uriEvEEEEN4mpl_2naESC_EENS5_INS1_3tagINS_12PropertyList8OrderTagESC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_EENS6_IS3_iXadL_ZNKS3_8getOrderEvEEEESC_EESC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_SC_EESaIS3_EEEi

gnash::iterator_find(boost::multi_index::multi_index_container<gnash::Property, boost::multi_index::indexed_by<boost::multi_index::ordered_unique<boost::multi_index::const_mem_fun<gnash::Property, gnash::ObjectURI const&, &(gnash::Property::uri() const)>, mpl_::na, mpl_::na>, boost::multi_index::ordered_unique<boost::multi_index::tag<gnash::PropertyList::OrderTag, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, boost::multi_index::const_mem_fun<gnash::Property, int, &(gnash::Property::getOrder() const)>, mpl_::na>, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na, mpl_::na>, std::allocator<gnash::Property> >&, int)

The second line is filtered through the binutils demangler to provide the decorated name of the function for C++. If you cannot guess, one of the problems is that the mangling algorithm contains no shorthand for boost templates, contrarily to what it has for the standard library templates. I know I cannot pretend that this relates directly with the problems I see with C++, but it shows two things: the first is that C++ has an unseen amount of complexity that even long-time developers fail to catch properly; the second is that ELF itself doesn’t seem well-designed to handle the way C++ is compiled into code; and this is all without even looking at the recent undocumented changes introduced in GLIBC to make sure that C++ is implemented correctly.

I wonder if I’ll ever be able to complete this demangler, and then start with the other schemes supported in ELF files…

Die grundsätzliche Verpflegung ist sichergestellt:
Sowohl Panera Bread (Frühstück und Kaffee), als auch das Outback Steakhaus und Joe’s Crab Shack (jeweils Abendbrot) sind gleich mehrfach vorhanden.

Gestern am Mittwoch war die Anreise nach Las Vegas. Ich habe mich doch entschieden, nicht mit Continental und nur einem Zwischenstopp zu fliegen. Anstelle dessen habe ich (wie fast immer) wieder Lufthansa/United mit zwei Zwischenstops gebucht. Erster Stop war in München, der anschließende Flug nach Chicago hat mich dann zwar wieder nördlich von Hamburg vorbeigeführt, aber von Hamburg direkt gibt es halt nicht so viele Flugmöglichkeiten. Die Strecke verlief weiter über Island und Grönland. Eine Aschewolke ist mir dabei aber nicht aufgefallen. Das kann natürlich auch daran gelegen haben, daß ich mit dem Mittagessen beschäftigt war. Es gab Salat mit Lachs zur Vorspeise und hervorragend gegrillte Filetspitzen zur Hauptspeise. Dazu ein etwas zu süßer kalifornischer Rotwein. Im Gegensatz zu meinem Sitznachbarn habe ich aber nicht die Gelegenheit ergriffen, mit ca. 15 Cola/Jim Beam schnell die Bettschwere zu erreichen.
Eigentlich hatte ich wieder nur auf einen schönen Platz am Notausgang oder ein kostenloses Upgrade in die Economy Plus spekuliert, welches es für Lufthansa Statuskunden häufig gibt. Aber bei einer überbuchten Economy habe ich mich gegen das Business-Upgrade auch nicht gewehrt …

In Chicago gab es dann noch gute zwei Stunden Extra-Aufenthalt wegen eines heftigen (aber sehr schönen) Gewitters und nach 26 Stunden Gesamtreisezeit bin ich in Las Vegas angekommen. Da wäre ich mit Continental vermutlich doch besser geflogen.

Der Bericht wird weitergeführt, sobald die Networkers begonnen hat …

Seems like people keep on expecting Ryan Gordon’s FatELF to solve all the problems. Today I was told I was being illogical by writing that it has no benefits. Well, I’d like to reiterate that I’m quite sure of what I’m saying here! And even if this is likely going to be a pointless expedient, I’ll try to convey once again why I think even just by discussing that we’re wasting time and resources.

I have to say, most of the people pretending that FatELF is useful seem to be expert mirror-climber, so they changed so many ideas on how it should be used, where it should be used, and which benefits it has, that this post will jump from point to point quite confusingly. I’m sorry about that.

First of all, let me try to make this clear: FatELF is going to do nothing to make cross-arch development easier. If you want easier cross-arch or cross-OS development, you go with interpreted or byte-compiled languages such as Ruby, Python, Java, C#/.NET, or whatever else. FatELF focuses on ELF files, which are produced by the C, C++, Fortran compilers and the like. I can’t speak for Fortran as it’s a language that I do not know, but C and C++ datatypes are very much specific to architecture, operating system, compiler, heck even version of the libraries, system and 3rdparty! You cannot solve those problems with FatELF, as whatever benefits it has, they can only appear after the build. But at any rate, let’s proceed.

FatELF supposedly make build easier, but it doesn’t. If you really think so you have never ever tried building something for Apple’s Universal Binary. Support for autoconf and most likely any other build system along those lines, simply suck. The problem is that whatever results you get from a test in one architecture might not have the same result in the other. And Apple’s Universal Binary only encompass an operating system that has been developed without thinking too much of compatibility with others, and was under the same tight controls, where the tests for APIs are going to be almost identical for all the arches. (You might not know, but Linux’s syscall numbers are not the same across architectures; the reason is that they are actually designed to partly maintain compatibility with proprietary (and non) operating systems that originated on that architecture and were mainstream at the time. So for instance on IA-64 the syscall numbers are compatible with HP-UX, while on SPARC are compatible with Solaris, for the most part.)

This does not even come close to consider the mess of the toolchain. Of course you could have a single toolchain patched to emit code for a number of architectures, but is that going to work at all? Given that I have actually worked as a consultant building cross-toolchains for embedded architectures, I can tell you that it’s difficult enough to get one working. Count the need for patches for specific architectures, and you might start to get part of a pictures. While binutils already theoretically supports a “multitarget” build that adds in one build the support for all the architectures that they have written code for, doing the same for gcc is going to be a huge mess. Now you could (as I suggested) write a cc frontend that takes care of compiling the code for multiple architectures at the time, but as I said above it’s not easy to ensure the tests are actually meaningful between architectures, let alone operating systems.

FatELF cannot share data sections. One common mistake to make thinking about FatELF is that it only requires duplication of executable sections (.text), but that’s not the case. Data sections (.data, .bss, .rodata) are dependent on the data types, which as I said above are architecture dependent, and operating system dependent, and even library dependent. They are part of the ABI; each ELF you build for a number of target arches right now is going to have its own ABI, so the data sections are not shared. The best I can think of, to reduce this problem, is to make use of -fdata-sections and then merge sections with identical content; it’s feasible, but I’m sure that at the best of cases is going to create a problem with caching of near objects, and the best it’s going to cause misalignment of data to be read in the same pass. D’uh!

Just so you know how variable are the data sections: even though you could just use #ifdef, both the Linux kernel and the GNU C Library (and most likely uClibc as well even though I don’t have it around to double-check it) install different sets of headers; this should be an indication of how different the interfaces are between them.

Another important note here: as far as I could tell from the specifics that Ryan provided (I really can’t be arsed to look back at them right now), FatELF files are not interpolated, mixing the sections of them, but merged in a sort-of archive, with the loader/kernel deciding which parts of it will be loaded as a normal ELF. The reason for this decision likely lies in one tiny winy detail: ELF files were designed to be mapped straight from disk to data structures in memory; for this reason, ELF have classes and data orders. For instance x86-64 uses ELF of class 64 and order LSB (Least-significant bit first, or little-endian) while PPC uses ELF of class 32 and order MSB (Most-significant bit first, or big-endian). It’s not just a matter of the content of .text but it’s also pervasive in the index structures within the ELF file, and it is so for performance reasons. Having to swap all the data or deal with different sizes is not something you want to do in the kernel loader.

When do you distribute a FatELF? This is one tricky question because both Ryan and various supporters change opinion here more often they change socks. Ryan said that he was working on getting an Ubuntu built entirely of FatELFs, leaving to intend that distributions would be using FatELF for their packaging. Then he said that it wasn’t something for packagers but for Indipendent Software Vendors (ISVs). Today I was told that FatELF simplifies the distribution by distributing a single executable that can be “executed in place” rather than having to provide two of them.

Let’s be clear: distributors are very unlikely to provide FatELF binaries in their packages. Even though it might sound tenting to implement multilib with them, it’s going to be a mess just the same; it might reduce the size of the whole archive of binary packages, because you share the non-ELF files between architectures, but it’ll increase the used traffic to download them, and while disk space is mostly getting cheaper and cheaper, network traffic is still a rare commodity. Even more so, users won’t like to have installed stuff for architectures that they don’t use, and will likely ask for a way to clean them up, at which point they’ll wonder why they are downloading it at all. Please note that while Apple did their best to convince people to use Universal Binary, a number of software was produced to strip the alternative architecture files from their executable files at all.

Today I was offered, as I said, that it is easier to distribute one executable file rather than two. But when are you found doing that at all? Quite rarely; ISVs provide pre-defined packaging, usually in form of binary packages for the particular distribution (this already makes it a multiple-file download). Most complex software will be in an archive anyway because you don’t just build everything in but rather put it in different files: icons, images, … even Java software that actually use archives as main object format (JAR files are ZIP files), ships in archives installing separate data files, multiple JARs, wrapper scripts and so on so forth. I was also told that you could strip the extra architectures at install time, but if you do so, you might as well decide which of multiple files to install, making it moot to use a fat binary at all.

All in all, I still have to see one use case that actually can be solved by FatELF better than a few wrapper scripts and an archive. Sure you can create some straw-man arguments where FatELF works and scripts don’t, such as the “execute in-place” idea above, but really tell me when was the last time you needed that? Please also remember that while “changes happen everytime”, we’re talking about changing in a particularly invasive way a number of layers:

• the kernel;
• the loader;
• the C library (separated from the loader!);
• the compiler, has almost to be rewritten;
• the linker, obviously;
• the tools to handle the files;
• all the libraries that change API among architectures.

Even if all of this became stock, there’s a huge marginal cost here, and it’s not going to happen anytime soon. And even if it did, how much time is going to take before it gets mainstream enough to be used by ISVs? There are some that sill support RHEL3.

There are “smaller benefits”, as, again, I was told before, and those are not nothing. Maybe that’s the case but the question is “is it worth it?”. Once in the kernel is not going to take much work at runtime, but is it work the marginal cost of implementing all that stuff and maintaining it? I most definitely don’t think so. I guess the only reason why Apple coped with that is that they had most of the logic code already developed and laying around from when they transitioned from M68K to PowerPC.

I’m sorry to burst your bubbles, but FatELF was an ill-conceived idea, that is not going to gain traction for a very good reason: it makes no sense! Any of the use-cases I have read up to now are straw-men, that either resemble what OSX does or what Windows does. But Linux is neither. Now, let’s move on, please?

I think this metaphor, extracted from a discussion between Lefty me and Carlo Piana, could really make it clear what my status is with respect to Free Software:

Have you got a washing machine? I guess you do; I’ll venture to say it’s a modern washing machine, let’s say.. from around 2005 or later. Good. Do you have the sources of the firmware of the dishwasher? I guess not, uh?

But I know there are people working on that, will give us a Free Software washing machine.

Sure, and once they’ll advertise washing machines with Free Software in them, hackable Free Software, by the way, I’ll gladly choose one as replacement when needed. Until then, I think I can still wash my clothes in the current, proprietary washing machine. It washes them just as fine, you know.

Now, Lefty goes a bit further questioning the usefulness of having free software on devices such as this one; on the other hand, I think it would still be a positive signal, as there are a few reason why free (hackable) firmware in those situations might be good for the user; on the other hand, it’s more than likely that it’ll hurt the profit margins of the vendors, so it’s not going to happen anytime soon.

For instance (this happened to me before, thus why I know about the situation is realistic), if you were to damage the logic board of your washing machine, it’s no longer just a matter of procuring yourself with the replacement component, like you did with older machinery. Even though there’ll be a number of shops to sell replacement components that are either compatible or even the original ones that left the manufacturer’s stock, they are shipped without firmware in them. So you need the firmware to make use of them.

Not only the firmware is proprietary, so getting a hold of it is illegal, but the firmware loaders themselves don’t store a copy of it any longer! They now switched to a set of flasher and 3G phone that downloads the firmware on-the-spot for a given model, and flash it right away on the board. You won’t have a copy of it, as it is.

With a Free Software (and hackable) firmware in the washing machine, instead, you’ll have the chance to simply take care of the flashing yourself if you ever had to replace the logic board, wouldn’t it be nice? And ecological as well since you wouldn’t replace the whole machine if the warranty ran out (replacement of logic board requires a technician call; it means that you can easily surpass the price of a new washing machine just by asking for the replacement). But it’s not just that; you could configure special washing routines, fine-tuned for the kind of clothes you wash, and the detergent you use.. or you could set it to only work during certain moments of the day.. all in all you’d have a terrific amount of choice in front of you!

But this utopia; manufacturers aren’t likely to give you access to your washing machine’s firmware; they have a business model going on with those replacement parts; they ask for more of your money to provide you with feature on your washing machines, even though these only usually come with sturdier, more capable (in form of its own hardware that is) machines that you might not have need for. Unless, of course, at some point a single manufacturer can find a way to produce low-cost decent-quality washing machines, that can give it more profit by selling the units than by struggling with replacement parts and technicians; at that point, a free, hackable firmware might make sense: take over the market by small, durable, tweakable yet affordable washing machines… and after that, the rest of the industry will have to follow suit as a “paradigm shift” started.

Who knows, it might happen. But until then, do you really think you should preach that Free Software users wash their clothes by hand? Or attack users and developers of proprietary washing machines “enemies of Free Software”? How’s this different from any other gadget? TV sets, dishwashers, phones (not just cellphones, your DECT has firmware as well, you know), you name it. And how are these different from specific software applications, or appliances if you prefer? My answer is “They aren’t”.

As long as I can accept the limitations I’m given, as long as it does not coerce me into something I don’t want to do, I’m happy to use the best tool, whatever that tool is, to complete a task. I have trust that such a tool is going to be, if not now in the future, Free Software. Not because I take that as the only important measurement, but because I know that the model works, and I have good reasons to prefer working with Free tools than not.

Einer der Kernpunkte in De Maizières Rede Grundlagen für eine gemeinsame Netzpolitik der Zukunft war ja das Thema Identität, denn er will ja den neuen elektronischen Personalausweis verkaufen und publiziert deswegen auch eine ganze Menge Angstpropaganda.

In Österreich hat man so etwas wohl schon seit einiger Zeit unter dem Namen Bürgerkarte (kann einer der hier mitlesenden Österreicher was dazu sagen?). Ebendie wird dort wohl großflächig mit Desinteresse bestraft, weil sie wohl in etwa so populär ist wie die üblichen digitalen Signaturkarten in Deutschland. Einer der Hauptkritikpunkte ist auch der, daß es zu einer Beweislastumkehr kommt, wenn man die digitale Signierfunktion in irgendeiner Weise verwendet, weil der ganze Prozeß zunächst einmal einen Anscheinsbeweis der Korrektheit hat. Das wird unter Umständen auch dem deutschen ePerso das Genick brechen.

In Österreich ruder man deswegen zurück, damit eGovernment überhaupt erst mal möglich wird, nach dem Motto 'Wenns gut genug ist für Banken, dann soll es dem Staat auch genügen".

In Deutschland soll der neue ePerso rechtzeitig zum Chaos Congress 2010 auf dem Markt sein.

Oli (from Dual): Ich denke, daß nur ca. 1 Promille der DB User so Zeugs wirklich braucht... aber heiß sind alle drauf. Die sollten besser mal SQL Tuning und Indexing lernen, statt mit so Zeugs rumzuspielen. Basics nicht im Griff, aber dann mit dem hochcomplexen Ding spielen. :)

• Ronald Bradford experimentiert mit MongoDB, einem Key-Value Store.
• Mehr MongoDB und Distributed Consistency.
• Mehr Distributed Consistency, sehr schöne Kurzerklärung. Schnabeltiere rocken.
• HBase vs. Cassandra und WTF is a SuperColumn (Cassandra Data Model).
• Digg setzt Cassandra ein, noch ein NoSQL Product. Dennis Forbes wundert sich wieso, auch in Reddit.
• Es gibt inzwischen SQL-Engines, die MapReduce und Sharding können. Closed Source und Closed Hardware: Clustrix und Open Source: VoltDB.
• Royal Pingdom listet die Technologie hinter Facebook auf.
• Artikelsammlung gegen ORM-Technologie.
• Obskures MySQL-Wissen: Outputting a Newline in SQL, sort_buffer_size und die immer wieder beliebte Frage in Kursen: mysql_use_result vs. mysql_store_result vs. HANDLER.

Ich muß mal wieder ein paar Tabs zu machen, zu denen ich eigentlich noch weitergehende Gedanken habe, aber bei denen ich in absehbarer Zeit nicht dazu kommen werde, das nachzurecherchieren und auszuformulieren.

• Die Meinungsmacher ist eine Rezension des gleichnamigen Buches von Leif Kramp und Stephan Weichert, die das symbiotische Zusammenspiel von Journalismus und Politik in Berin analysiert und kontextualisiert.
• Mercedes Bunz faßt in ihrem Blog unter dem Titel How Google plans to save the news einen längeren, sehr lesenswerten Artikel von James Fallows zusammen.
• Was, wenn Qualitätsjournalisten ein Thema aus Blogs aufgreifen und medial verwursten, ohne die Originalquelle zu entlohnen oder gar zu nennen? The Thought Copier beim Hyperlocalisten und How The Mainstream Media Stole Our News Story Without Credit beschreiben die Situation und den Umgang damit, an einem Beispiel.
• Robert Fisk, The Independent, schreibt in Al Jazeera (English Edition) über den Umgang von Journalistem mit Spin in der Sprache, Journalism and 'the words of power' ist ätzend, unangenehm genau und auf den Punkt.
• 82% Of Users Will Abandon Their Favorite News Site If They Put Up A Paywall: Sind Benutzer bereit, für eine Zeitungs-Website zu bezahlen?

Unser Innenminister hat Angst. Vor dem Internet. Für ihn ist das eine nichtstaatliche Öffentlichkeit, die sich ohne Staat organisiert und ohne Staat funktioniert. Und ganz besonders ohne den deutschen Staat. Darum hat er sich in einer Grundsatzrede zum Thema Internet geäußert und 14 Thesen formuliert, die am Ende auf mehr Regulierung, mehr Eingriffe, mehr Haftung, weniger Anonymität und weniger Freiheit hinauslaufen.

Der Providerverband eco hat das schon erkannt, und wir sollten uns auch Sorgen machen.

Lately I got a number of new requests about the status of LXC (Linux Containers) support in Gentoo; I guess this is natural given that I have blogged a bit about it and my own tinderbox system relies on it heavily to avoid polluting my main workstation’s processes with the services used by the compile – and especially test – phases. Since a new version was released on Sunday, I guess I should write again on the subject.

I said before that in my opinion LXC is not ready yet for production use, and I maintain that opinion today. I would also rephrase it in something that might make it easier to understand what I think: I would never trust root on a container to somebody I wouldn’t trust root with on the host. While it helps a great deal to reduce the nasty effects of an application mistakenly growing rogue, it neither removes the option entirely, nor it strengthen the security for intentional meddling with the system. Not alone at least. Not as it is.

The first problem is something I have already complained about: LXC shares the same kernel, obviously and by design; this is good because you don’t have to replicate drivers, resources, additional layers for filesystem and all the stuff, so you have real native performance out of it; on the other hand, this also means that if the kernel does not provide namespace/cgroup isolation, it does not allow you to make distinct changes between the host system and the container. For instance, the kernel log buffer is still shared among the two, which causes no little problems to run a logger from within the container (you can do so, but you have to remember to stop it from accessing the kernel’s log). You also can’t change sysctl values between the host and the container, for instance to disable the brk() randomizer that causes trouble with a few LISP implementations.

But there are even more interesting notes that make the whole situation pretty interesting. For instance, with the latest release (0.7.0), networking seems to have slightly slowed down; I’m not sure what’s the problem exactly, but for some reason it takes quite a bit longer to connect to the container than it used to; nothing major so I don’t have to pay excessive attention to it. On the other hand, I took the chance to try again to make it work with the macvlan network rather than the virtual Ethernet network, this time even googling around to find the solution about my problem.

Now, Virtual Ethernet (veth) is not too bad; it creates a peer-to-peer connection between the host and the container; you can then manage that as you see fit; you can then set up your system as a router, or use Linux ability to work as a bridge to join container’s network with your base network. I usually do that, since it reduces the amount of hops I need to add to reach Internet. Of course, while all the management is done in-kernel, I guess there are a lot of internal hops that have to be passed, and for a moment I thought that might have been slowing down the connection. Given that the tinderbox accesses the network quite a bit (I use SSH to control it), I thought macvlan would be simpler: in that case, the kernel is directing the packets coming toward a specific MAC address through the virtual connection of the container.

But the way LXC does it, it means that it’s one-way. By default, actually, each macvlan interface you create, isolates the various containers one from the other as well; you can change the mode to “bridge” in which case the containers can chat one with the other, but even then, the containers are isolated from the host. I guess the problem is that when they send packets, they get sent out from the interface they are bound to but the kernel will ignore them if they are directed back in. No there is currently no way to deal with that, that I know of.

Actually upstream has stated that there is no way to deal with that right now at all. Sigh.

An additional problem with LXC is that even when you do blacklist all the devices so that the container’s users don’t have access to the actual underlying hardware, it can mess up your host system quite a bit. For instance, if you were to start and stop the nfs init script inside the container.. you’d be disabling the host’s NFS server.

And yes, I know I have promised multiple time to add an init script to the ebuild; I’ll try to update it soonish.

I recently found out that my cheap DSL modem (Allnet 0333B) which i bought because the one that was provided by my ISP broke down, has a configuration page which shows the line status. It can easily access by adding a private ip to the wan interface of the router. Execute the the following on your router (e.g. via ssh or execute form on the webinterface)
ifconfig `nvram get wan_ifname`:0 172.16.1.250 netmask 255.255.255.0
iptables -t nat -I POSTROUTING -o `nvram get wan_ifname` -j MASQUERADE
Now the modem can be reached from the LAN:

~# ping 172.16.1.254 -c 1
PING 172.16.1.254 (172.16.1.254) 56(84) bytes of data.
64 bytes from 172.16.1.254: icmp_req=1 ttl=254 time=26.2 ms

--- 172.16.1.254 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 26.211/26.211/26.211/0.000 ms

http://172.16.1.254:8235/cgi-bin/webcm

admin / coolwhite

telnet://172.16.1.254
root / coolwhite

(can be obtained from the GPL sources)

The funny thing is, the officially as modem announced device can be configured as full router with advanced features like portforwarding, access control, QoS etc.

last but not least, my (poor) line status:

A “fellow” Gentoo developer today told me that we shouldn’t try to get --as-needed working because it’s a task for upstream (he actually used the word “distributors” to intend that neither Gentoo nor any other vendor should do that, silly him)… this developer will go unnamed because I’ll also complain right away that he suggested I give up on Gentoo when I gave one particular reason (that I’ll repeat in a moment) for us to do that instead. Just so you know, if it was up to me, that particular developer right now would have his access revoked. Luckily for him, I have no such powers.

Anyway, let me try to put in proper writing why it should fall to Gentoo to enable that by default to protect our users.

The reason I gave above (it was a twitter exchange so I couldn’t articulate it completely), is that “Fedora does it already”. To be clear, both Fedora and Suse/Novell do it already. I’m quite sure Debian doesn’t do it, and I guess Ubuntu doesn’t do that either to keep in line with Debian. And the original hints we took to start with --as-needed came from AltLinux. This alone means that there is quite a bit of consensus out there that it should be a task for distributors to look at. And it should say a lot that the problems solved by --as-needed are marginal for binary distributions like the ones I named here; they all do that simply to reduce the time needed for their binary packages to rebuild, rather than to avoid breaking users’ systems!

But I’m the first person to say that the phrase “$OtherDistribution does it, why don’t you?” is bogus and actually can cause more headaches than it solves. Although most of the time, this is meant when $Otherdistribution is Ubuntu or a relative of theirs. I seriously think that we should take a few more hints from Fedora; not clone them, but they have very strong system-level developers working on their distributions. But that’s a different point altogether by now.

So what other reasons are there for us to provide --as-needed rather than upstream? Well, actually this is the wrong question; the one you should formulate is “Why is it not upstream’s task to use --as-needed?”. While I have been pushing --as-needed support in a few of my upstream packages before, I think by now that it’s not the correct solution. It all boils down to who knows better whether it’s safe to enable --as-needed or not. There are a few things you should assess before enabling --as-needed:

• does the linker support --as-needed? it’s easier said than done; the linker might understand the flag, but supporting is it another story; there are older versions of ld still in active use that will crash when using it; other with a too greedy --as-needed that will drop libraries that are needed, and only recently the softer implementation was added; while upstream could check for a particualr ld version, what about backports?
• do the libraries you link to, link to all their needed dependencies? one of the original problems with --as-needed when introduced to the tree was that you’d have to rebuild one of the dependencies because it relied on transitive linking, which --as-needed disallowed (especially in its original, un-softened form); how can a given package make sure that its dependencies are all fine before enabling --as-needed?
• do the operating system at all support --as-needed? while Gentoo/FreeBSD uses modern binutils, and (most of) the libraries are built so that all the dependencies are linked in for --as-needed support, using it is simply not possible (or wasn’t possible at least…), because gcc will not allow for linking the threading libraries in for compatibility with pre-libmap.conf FreeBSD versions; this has changed recently for Gentoo since we only support more recent versions of the loader that don’t have that limitation; even more so, how can upstream know whether the compiler will have the fix already or not?

Of course you can “solve” most of the doubts by running runtime tests; but is that what upstreams should do? Running multiple tests from multiple packages require sharing the knowledge and risks for the tests to get out-of-sync one with the other; you have redundancy of work.. when instead the distributor can simply decide on whether using --as-needed is part of their priorities or not. It definitely is for Fedora, Suse, AltLinux… it should be for Gentoo as well, especially as a source-based distribution!

Of course, you can find some case-by-case where --as-needed will not work properly; PulseAudio is unfortunately one of those, and I haven’t had the time to debug binutils to see why the softer rules don’t work well in that case. But after years working on this idea, I’m very sure that it’s a very low percentage of stuff that fails to work properly with this, and we should not be taken hostage by a handful of packages out of over ten thousands!

But, when you don’t care about the users’ experience, when you’re just lazy or you think your packages are “special” and deserve to break any possible rule, you can afford yourself to ignore --as-needed. Is that the kind of developers Gentoo should have, though? I don’t think so!

The title of this post seems a bit messed up, but it’ll make sense at the end. It’s half a recount of my persona hardware trouble and half a recount of my fighting with Apple’s software, and not of the kind my reader hate to read about I guess.

I recently had to tear apart my Seagate FreeAgent Xtreme external HDD. The reasons? Well, beside leaving me without a connection while using it (with Yamato) on eSATA, and forcing me to use either Firewire or USB (both much slower — and I did pay it to use eSATA!), yesterday it decided it didn’t want to let me access anything via either of the three connections, not even after a number of power cycles (waiting for it to cool down as well); this was probably related to the fact that I tried to use it again as eSATA, connected to the new laptop to try copying an already set-up partition from the local drive to make space for (sigh) Windows 7.

Luckily, there was no data worth spending time on, in that partition, just a few GNOME settings I could recreate in a matter of minutes anyway.

Since the Oxford Electronics-based bridge on the device decided not to help me out to get my data back, I decided to break it up, with the help of a Youtube video (don’t say that Youtube isn’t helpful!), and took the drive itself out, which is obviously a Seagate 7200.11 1TB drive, quite a sturdy one to look at it. No I won’t add it at the 7th disk drive to Yamato, mostly because I fear it wouldn’t be able to start up anymore if I did so.

Thankfully, I bought a Nilox-branded “bay” a month or so ago, when I gave away what remained of Enterprise to a friend of mine (the only task that Enterprise was still doing was saving data out of SATA disks when people brought me laptops or PCs that fried up. My choice for that bay was due to the fact that it allows you to plug in both 3.5" and 2.5" SATA disks without having to screw them anywhere. It does look a lot like something out of the Dollhouse set, to be honest, but that doesn’t matter now.

I plugged it in, and started downloading the data; I can’t be sure it is all fine, so I deleted lots and lots of stuff I won’t be safe about for a while. Then I shivered, fearing the disk itself was bad, and that I had no way to check it out… thankfully, the bay uses Sunplus electronics in it, and, lo and behold!, smartmontools has a driver for the Sunplus USB bridge! A SMART test later, and the disk turns out to feel better than any other disk I ever used. Wow. Well, it’s expected as I never compiled on it.

Anyway, what can I do with a 1TB SATA disk I cannot plug into any computer as it is? Well, actually one thing I can do: backup storage. Not the kind of rolling backup I’m currently doing with rsnapshot and the WD MyBook Studio II in eSATA (anything else is just too slow to backup virtual machines), but rather a fixed backup of stuff I don’t expect to be looking at or using anytime soon. But to be on the safe side, I wanted to have it available in a format I can access, on the go, with the Mac as well as from Linux; and vfat is obviously not a good choice.

The choice is, for the Nth time, HFS+. Since Apple has published quite a bit of specs on the matter, the support in Linux is decent, albeit far from being perfect (I still haven’t finished my NFS export patch, it does not support ACLs or extended attributes, and so on). It’s way too unreliable for rsnapshot (with hardlinking) but should work acceptably well for the storage.

The only reason I have not to use it for something I want to rely on, as it is, is that the tools for filesystem creationa nd check (mkfs and fsck) are quite a bit old. I’m not referring to “hfsutils” or “hfsplusutils” both of which are written from scratch and have a number of problems, including but not limited to, shitty 64-bit code. I’m referring to the diskdev_cmds package in Gentoo which is a straight port of Apple’s own code, which is released as FLOSS under the APSL2 license.

Yes, I call that FLOSS! You may hate Apple as much as you wish, but even FSF considers APSL2 a Free Software license albeit with problems; on the other hand they explicitly state this (emphasis mine):

For this reason, we recommend you do not release new software using this license; but it is ok to use and improve software which other people release under this license.

Anyway, I went to Apple’s releases for 10.6.3 software (interestingly they haven’t published yet 10.6.4 which was released just the other day), and downloaded diskdev_cmds, and the xnu package that contains their basic kernel interfaces, and I started working on an autotools build system to make it possible to easily port the code in the future (thanks to git and branching).

The first obstacle, beside the includes obviously changing, was that Apple decided to make good use of a feature they implemented as part of Snow Leopard’s “Grand Central Dispatch”, their “easy” multi-threading implementation (somewhat similar to the concept of OpenMP): “blocks”. Anonymous functions for the C language, an extension they worked in LLVM. So GCC straight is unable to build the new diskdev_cmds. I could either go to fetch an older diskdev_cmds tarball, from Leopard rather than Snow Leopard, where GCD was not implemented yet, or I could up the ante and try to get it working with some other tools. Guess what?

In Gentoo we already have LLVM around, and the clang frontend as well. I decided to write an Autoconf check for blocks support, and rely on clang for the build. Unfortunately it also needs Apple’s own libclosure, that provides some interfaces to work with blocks. And the basis for the GDC interface. It actually resonated a bit when Snow Leopard was presented because Apple released it for Windows as well, with the sources under MIT license (very liberal). Unfortunately you cannot find it in the page I linked above but you have to look at 10.6.2 page for whatever reason.

I first attempted to merge this straight in the diskdev_cmds sources, but then I guessed that it makes more sense to try porting it alone, and make it available, maybe somebody will find some good use for it. Unfortunately the task is not as trivial as it looks. The package needs two very simple functions for “atomic compare and swap” which OS X provides as part of its base library, and so does Windows. On Linux, equivalent functions are provided by HP’s libatomic_ops (you probably have it around because of PulseAudio).

Unfortunately, libatomic_ops does not build, as it is, with clang/LLVM; there is a mistake in the code, or the way it’s parsed; it’s not something unexpected given that inline assembler is a lot compiler-dependent. In this case it’s a size problem: it uses a constraint for integer types (32-bit) but a temporary (and same-sized input) of type unsigned character (8-bit). The second stop is again libatomic_ops’s problem: while it provides an equivalent interface to do atomic compare and swap for long types, it doesn’t do so for int types; that means it works fine on x86 (and other 32-bit architectures where both types are 32-bit) but it won’t do for x86-64 and other 64-bit architectures. Guess what the libclosure code needs?

Now of course it would be possible to lift the atomic operations out of the xnu code, or just write them straight, as libatomic_ops already provides them all, just not correctly-sized for x86-64 but the problem remains that you then have to add a number of functions for the various architecture rather than having a generic interface; xnu provides functions only for x86/x86-64 and PPC (since that’s what Apple uses/used).

And where has this left me now? Well, nowhere far, mostly with a sour feeling about libatomic_ops inability to provide a common, decent interface (for those who wonder, they do provide char-sized inlines for compare and swap for most architecture, and even the int-sized alternatives that I was longing for… but only for IA-64. You wouldn’t believe that until you remembered that the whole library is maintained by HP.

If I could take the time off without risking trouble, I would most likely try to get better HFS+ support in Linux, if only to make it easier and less troublesome for OSX users to migrate to Linux at one point or another. The specs are almost all out there, the code as well. Unfortunately I’m no expert in filesystems and I lack the time to invest on the matter.

DMVPN ist eine der elegantesten VPN-Arten, wenn man eine größere Anzahl von Außenstellen hat, die über das Internet verbunden sind. Leider ist die Konfiguration von Quality of Service (QoS) hierbei leicht eingeschränkt. Vor einiger Zeit habe ich es bei einem ersten Kunden gewagt, Per-Tunnel QoS zu konfigurieren. Gewagt deshalb, weil ich dafür von meinem Lieblings-IOS, 12.4(15)T, auf ein neueres IOS wechseln musste. Im aktuellen Fall hat sich 12.4(22)T5 als stabil und zuverlässig genug herausgestellt. Die vorher getestete Version 12.4(22)T4 hatte massive Speicherlecks und war noch nicht einsetzbar.

Was ist Per-Tunnel-QoS?
Beim DMVPN registriert sich der Spoke-Router per NHRP, um seine (dynamische) public IP beim Next-Hop-Server (NHS) zu registrieren. Dabei kann der Router gleich einen Gruppennamen mitgeben, der in der Per-Tunnel-QoS-Konfiguration verwendet wird. Auf dem Hub wird für jede Gruppe eine QoS-Konfiguration angewendet. Damit lässt sich z.B. für jeden Spoke-Router die Datenrate auf die in der Außenstelle verwendete Downstream-Geschwindigkeit shapen. Wenn in der Hauptstelle eine Internet-Anbindung mit 34 MBit/s vorhanden ist, würde man damit normalerweise fast jede Außenstelle überlasten. Mit Per-Tunnel-QoS lässt sich das individuell runter regeln und wichtiger Traffic kann auch priorisiert werden.

Die Spoke-Konfiguration
Diese beschränkt sich auf eine Zeile in der Konfiguration:

interface Tunnel1
  description DMVPN-Tunnel Internet
  ...
  ip nhrp group SDSL2000-Std
  ...

Der Gruppenname (hier SDSL2000-Std) wird auf der Hub-Seite für die Auswahl der richtigen Policy verwendet. In diesem Beispiel ist die Außenstelle mit einer 2 MBit/s SDSL-Leitung angebunden, in der kein Voice verwendet wird. Die 2 MBit/s-Außenstelle mit Voice benutzt dann z.B. den Gruppennamen SDSL2000-Voice.

Die Gruppen sind auf dem Hub sichtbar:

HUB-Router#sh ip nhrp
...
10.255.255.8/32 via 10.255.255.8
   Tunnel1 created 2w5d, expire 01:43:49
   Type: dynamic, Flags: registered used
   NBMA address: 79.x.y.z
   Group: SDSL2000-Std
10.255.255.9/32 via 10.255.255.9
   Tunnel1 created 20:53:37, expire 01:46:01
   Type: dynamic, Flags: registered
   NBMA address: 88.x.y.z
   Group: HSDPA-Std
10.255.255.16/32 via 10.255.255.16
   Tunnel1 created 2w5d, expire 01:54:05
   Type: dynamic, Flags: registered used
   NBMA address: 80.x.y.z
   Group: SDSL2000-Std
10.255.255.152/32 via 10.255.255.152
   Tunnel1 created 2w5d, expire 01:29:46
   Type: dynamic, Flags: registered
   NBMA address: 80.x.y.z
   Group: ADSL3000-Voice
...

Die Hub-Konfiguration
Auf dem Hub wird als Minimum eine Policy für Shaping konfiguriert, die von der NHRP-Gruppe abhängig ist:

policy-map SDSL2000-Std-Parent
 class class-default
    shape average 2000000
policy-map HSDPA-Std-Parent
 class class-default
    shape average 3000000

Diese Policy-Map wird im DMVPN-Tunnel an die Ziel-NHRP-Gruppe gebunden:

interface Tunnel1
  description DMVPN-Tunnel Internet
  ip nhrp map group HSDPA-Std service-policy output HSDPA-Std-Parent
  ip nhrp map group SDSL2000-Std service-policy output SDSL2000-Std-Parent

Für alle Standorte, die diese NHRP-Gruppen verwenden, wird die ausgehende Datenrate auf zwei, bzw. auf drei MBit/s begrenzt.
Für die Standorte, die auch Voice benutzen, muss innerhalb dieser begrenzten Datenrate aber der Voice-Traffic bevorzugt werden. Dafür wird eine hierarchische Policy-Map konfiguriert:

class-map match-all EF-TRAFFIC
  match  dscp ef
!
policy-map ADSL3000-Voice
 class EF-TRAFFIC
    priority 256
 class class-default
    fair-queue
policy-map ADSL3000-Voice-Parent
 class class-default
    shape average 3000000
  service-policy ADSL3000-Voice

Hier wird in der Parent-Policy der Traffic auf 3 MBit geshaped. In diesen drei MBit/s wird 256 kBit/s für Voice-Traffic priorisiert. Auch das weitere gewünschte QoS-Verhalten würde in der Child-Policy konfiguriert werden. In dem Tunnel-Interface wird dann auch hier die Parent-Policy an die NHRP-Gruppe gebunden:

 interface Tunnel1
  ip nhrp map group ADSL3000-Voice service-policy output ADSL3000-Voice-Parent

Die Wirkung der QoS-Implementierung kann man dann mit “show dmvpn detail” und “show policy-map multipoint” überprüfen:

HUB-Router#sh dmvpn detail
...
    1 80.x.y.z     10.255.255.16    UP 20:38:56    D     10.255.255.16/32
NHRP group: SDSL2000-Std
 Output QoS service-policy applied: SDSL2000-Std-Parent

    1  80.x.y.z    10.255.255.152    UP     2w5d    D    10.255.255.152/32
NHRP group: ADSL3000-Voice
 Output QoS service-policy applied: ADSL3000-Voice-Parent

HUB-Router#sh policy-map multipoint 

Interface Tunnel1 --> 80.x.y.z

  Service-policy output: ADSL3000-Voice-Parent

    Class-map: class-default (match-any)
      3643772 packets, 1136319199 bytes
      5 minute offered rate 0 bps, drop rate 0 bps
      Match: any
      Queueing
      queue limit 750 packets
      (queue depth/total drops/no-buffer drops) 0/0/0
      (pkts output/bytes output) 3744389/1409612078
      shape (average) cir 3000000, bc 12000, be 12000
      target shape rate 3000000

      Service-policy : ADSL3000-Voice

        queue stats for all priority classes:
          Queueing
          queue limit 64 packets
          (queue depth/total drops/no-buffer drops) 0/0/0
          (pkts output/bytes output) 1046668/312237672

        Class-map: EF-TRAFFIC (match-all)
          1045840 packets, 242902872 bytes
          5 minute offered rate 0 bps, drop rate 0 bps
          Match:  dscp ef (46)
          Priority: 256 kbps, burst bytes 6400, b/w exceed drops: 0

        Class-map: class-default (match-any)
          2597932 packets, 893416327 bytes
          5 minute offered rate 0 bps, drop rate 0 bps
          Match: any
          Queueing
          queue limit 686 packets
          (queue depth/total drops/no-buffer drops/flowdrops) 0/0/0/0
          (pkts output/bytes output) 2697721/1097374406
          Fair-queue: per-flow queue limit 171
...

Verbleibende potentielle Probleme:
Auch Per-Tunnel QoS löst natürlich nicht alle Probleme. Wenn man z.B. mehrere Hubs hat, die Traffic zu den Spokes senden, dann weiß Hub1 nicht, wieviel Traffic Hub2 sendet. Auch haben die Hubs keine Information, ob die Spokes evtl. durch lokalen Internet-Traffic oder aber durch Spoke-to-Spoke-Kommunikation überlastet sind. Trotzdem kann die Kommunikation mit diesem Modell in vielen Fällen optimiert werden.

Zusätzlich werden einzelne Pakete durch das Shaping evtl. länger zurückgehalten, wodurch sich die Reihenfolge der IPSec-Pakete ändern kann. Der Replay-Buffer der Router muss also deutlich vergrößert oder der Replay-Check gar komplett ausgeschaltet werden:

crypto ipsec security-association replay window-size 1024
no crypto ipsec security-association replay window-size

This post is brought to you by a conversation with Fabio, which actually reminded me of an older conversation I had with someone else (exactly whom, right now, escapes me) about the ability to inject RPATH values into already-built binaries. I’m sorry to have forgotten who asked me about that, I hope he or she won’t take it bad.

But before I go to the core of the problem, let me try to give a brief introduction of what we’re talking about here, because jumping straight to talk about injection is going to be too much for most of my readers, I’m sure. Even though, this whole topic reconnects with multiple other smaller topics I discussed about in the past on my blog, so you’ll see a few links here and there for that.

First of all, what the heck is RPATH? When using dynamic linking (shared libraries), the operating system need to know where to look for the libraries an executable uses; to do so, each operating system has one or more PATH variables, plus eventual configuration files, that are used to look up the libraries; on Windows, for instance, the same PATH variable that is used to find the commands is used to load the libraries; and the libraries are looked for in the same directory where the executable is first of all. On Unix, the commands and the libraries use distinct paths, by design, and the executable’s directory is not searched for; this is also because the two directories are quite distinct (/usr/bin vs /usr/lib as an example). The GNU/Linux loader (and here the name is proper, as the loader comes out of GLIBC — almost identical behaviour is expected by uClibc, FreeBSD and other OSes but I know that much for sure) differs extensively from the Sun loader; I say this because I’ll introduce the Sun system later.

In your average GNU/Linux system, including Gentoo, the paths where to look up the libraries in are defined in the /etc/ld.so.conf file; prepended to that list, the LD_LIBRARY_PATH variable is used. (There is a second variable, LDPATH LIBRARY_PATH, that tells the gcc frotnend to the linker where to look for libraries to link to at build time, rather than to load — update: thanks to Fabio who pointed me I got the wrong variable; LDPATH is used by the env-update script to set the proper search paths in the ld.so.conf file ). All the executables will look in all these paths for both the libraries they link to directly and for non-absolute dlopen() calls. But what happens with private libraries — libraries that are shared only among a small number of executables coming from the same package?

The obvious choice is to install them normally in the same path as the general libraries; this does not require playing with the search paths at all, but it causes two problems: the build-time linker will still find them during link time, and it might not be what you want and it increases the number of files present in the single directory (which means accessing its content slows down, little by little). The common alternative approach is installing it in a sub-directory that is specific to the package (automake already provides a pkglib installation class for this type of libraries). I already discussed and advocated this solution so that internal libraries are not “shown” to the rest of the software on the system.

Of course, adding the internal library directories to the global search path also means slowing down the libraries’ load, as more directories are being searched when looking for the libraries. To solve this issue, runpath first and rpath now is used. The DT_RPATH is a .dynamic attribute that provides further search paths for the object it is listed in (it can be used both for executables and for shared objects); the list is inserted in-between the LD_LIBRARY_PATH environment variable and the search libraries from the /etc/ld.so.conf file. In that paths, there are two special cases: $ORIGIN is expanded to be the path of the directory where the object is found, while both empty and . values in the list are meant to refer to the current work directory (so-called insecure rpath).

Now, while rpath is not a panacea and also slightly slow down the load of the executable, it should have decent effects, especially by not requiring further symlinks to switch among almost-equivalent libraries that don’t have ABIs that stable. They also get very useful when you want to build a software you’re developing so that it loads your special libraries rather than the system one, without relying on wrapper scripts like libtool does.

To create an RPATH entry, you simply tell it to the linker (not the compiler!), so for instance you could add to your ./configure call the string LDFLAGS=-Wl,-rpath,/my/software/base/my/lib/.libs to build and run against a specific version of your library. But what about an already-linked binary? The idea of using RPATHs make it also for a nicer handling of binary packages and their dependencies, so there is an obvious advantage in having the RPATH editable after the final link took place… unfortunately this isn’t as easy. While there is a tool called chrpath that allows you to change an already-present RPATH, and especially to delete one (it comes handy to resolve insecure rpath problems), it has two limitations: the new RPATH cannot be longer than the previous one, and you cannot add a new one from scratch.

The reason is that the .dynamic entries are fixed-sized; you can remove an RPATH by setting its type to NULL, so that the dynamic loader can skip over it; you can edit an already-present RPATH by changing the string it points to in the string table, but you cannot extend neither .dynamic nor the string table itself. This reduces the usefulness of RPATH for binary package to almost nothing. Is there anything we can do to improve on this? Well, yes.

Ali Bahrami at Sun already solved this problem in June 2007, which means just over three years ago. They implemented it with a very simple trick that could be implemented totally in the build-time linker, without having to change even just a bit of the dynamic loader: they added padding!

The only new definition they had to add was DT_SUNW_STRPAD, a new entry in the .dynamic table that gives you the size of the padding space at the end of the string table; together with that, they added a number of extra DT_NULL entries in the same .dynamic table. Since all the entries in .dynamic are fixed sized, a DT_NULL can become a DT_RPATH without a problem. Even if some broken software might expect the DT_NULL at the end of the table (which would be wrong anyway), you just need to keep them at the end. All the ELF software should ignore the .dynamic entries they don’t understand, as long as they are in the reserved specific ranges, at least.

Unfortunately, as far as I know, there is no implementation of this idea in the GNU toolchain (GNU binutils is where ld is). It shouldn’t be hard to implement, as I said; it’s just a matter of emitting a defined amount of zeros at the end of a string table, and add a new .dynamic tag with its size… the same rpath command from OpenSolaris would probably be usable on Linux after that. I considered porting this myself, but I have no direct need for it; if you develop proprietary software for Linux, or need this feature for deployment, though, you can contact me for a quote on implementing it. Or you can do it yourself or find someone else.

I’m still not sure why I work with Rails, if it’s really for the quality of some projects I found, such as Typo, if it is because I hate most of the alternatives even more (PHP, Python/Turbogears/ToscaWidgets), or because I’m masochist.

After we all seemed to settle in with Rails 2.3.5 as the final iteration of the Rails 2 series, and were ready to face a huge absurd mess with Rails 3 once released, the project decided to drop a further bombshell on us in the form of Rails 2.3.6, and .7, and finally .8 that seemed to work more or less correctly. These updates weren’t simple bugfixes, because they actually went much further: they changed the supported version of Rack from 1.0.1 to 1.1.0 (it changes basically the whole internal engine of the framework!), the version if tmail and of i18n. It also changed the tzinfo version, but that’s almost pointless when considered in Gentoo since we actually use it unbundled and keep it up-to-date when new releases are made.

But most likely the biggest trouble with the new Rails version is implementing an anti-XSS interface compatible with Rails 3; this caused quite a stir because almost all Rails applications needed to be adapted for that to work. Typo for instance is still not compatible with 2.3.8, as far as I know! Rails-extensions and other Rails-tied libraries also had to be updated, and when we’ve been lucky enough, upstream kept them compatible with both the old and the new interfaces.

At any rate, my current job requires me to work with Rails and with some Rails extensions; and since I’m the kind of person who steps up to do something more even though it’s not paid for, I made sure I had ebuilds, that I could run test with, for all of them. This actually turned out more than once useful, and as it happens, today was another of those days when I’m glad I’m developing Gentoo.

The first problem appeared when it came time to update to the new (minor) version of oauth2 (required to implement proper Facebook-connected login with their changes last April); for ease of use with their Javascript framework, the new interface uses almost exclusively the JSON format; and in Ruby, there is no shortage of JSON interpreters. Indeed, beside the original implementation in ActiveSupport (part of Rails) there is the JSON gem, which provides both a “pure ruby” implementation and a compiled implementation (to be honest, the compiled, C-based implementation, was left broken in Gentoo for a while by myself, I’m sorry about that and as soon as I noticed I corrected it); then there is the one I already discussed briefly together with the problems related to Rails 2.3.8: yajl-ruby. Three is better than one, no? I beg you to differ!

To make it feasible to choose between the different implementations as wanted, the oauth2 developers created a new gem, multi_json, that allows to switch between the different implementations. No it doesn’t even try to give a single compatible exception interface, so don’t ask, please. It was time to pack the new gem then, but that caused a bit of trouble by itself: beside the (unfortunately usual) trouble with the Rakefile demanding RSpec to even just declare the targets (so also to build documentation) or the spec target having a dependency over the Jeweler-provided check_dependencies, the testsuite failed on both Ruby 1.9 and JRuby quite soon; the problem? It forced testing with all the supported JSON engines; but Ruby 1.9 lacks ActiveSupport (no, Rails 2.3.8 does not work with Ruby 1.9, stop asking), and JRuby lacks yajl-ruby since that’s a C-based extension. A few changes later and the testsuite reports pending tests when the tested engine is not found as it should have from the start. But a further problem appears in the form of oauth2 test failures: the JSON C-based extension gets identified and loaded but the wrong constant is used to load the engine, which results in multi_json to crap on itself. D’oh!

This was already reported on the GitHub page, on the other hand I resolved to fix it in a different way, possibly more complete; unfortunately I didn’t get it entirely right because the order in which the engines are tested is definitely important to upstream (and yes it seems to be more of a popularity contest than an actual technical behaviour contest, but nevermind that for now). Fixed that, another problem with oauth2 appeared, and that turned out to be caused by the ActiveSupport JSON parser; while the other parsers seems to validate the content they are provided with, AS follows the “Garbage-in, Garbage-out” idea: it does not give any exception if the content given is not JSON; this wouldn’t be so bad if it wasn’t that the oauth2 code actually relied on this to be able to choose between JSON and Form-Encoded parameters given. D’oh! One more fix.

Speaking about this, Headius if you’re reading me you should consider adding a pure-Java JSON parser to multi_json just for the sake of pissing off MRi/REE guys… and to provide a counter-test of a not-available engine.

At least, the problems between multi_json and oauth2 had the decency to happen only when non-standard setup were used (Gentoo is slightly non-standard because of auto_gem), so it’s not entirely upstream’s fault to not have noticed them. Besides, kudos to Michael (upstream) who already released the new gems while I was writing this!

Another gem that I should be using and was in need of a bump was facebooker (it’s in the ruby-overlay rather than in main tree); this one was already bumped recently for compatibility with Rails 2.3.8, but keeping itself compatible with the older release, luckily. Fixing the testsuite for the 1.0.70 version has been easy: it was forcing Rails 2.3.8 in the lack of multi_rails, but I wanted it to work with 2.3.5 as well, so I dropped that forcing. With the new release (1.0.71) the problem became worse because the tests started to fail. I was afraid the problem was in dropped 2.3.5 compatibility but that wasn’t the case.

First of all, I worked out properly the problem of forcing 2.3.8 version of Rails by making it abide to RAILS_VERSION during testing; this allows me not to edit the ebuild for each new version, as I just have to tinker with the environment variables to get them right. Then I proceeded with a (not too short) debugging session, which finally catapulted me to a change in the latest version. Since one function call now fails with Rack 1.1.0 for non-POST requests, the code was changed to ignore the other requests; the testsuite on the other hand tested with both POST and GET requests (which makes me assume that it should work in both cases). The funny part was that the (now failing) function above was only used to provide a parameter to another method (which then checked some further parameters)… but that particular argument was no longer used. So my fix was to remove the introduced exclusion for non-POST, remove the function call, and finally remove the argument. Voilà, the testsuite now passes all green.

All the fixes are in my github page and the various upstream developers are notified about them; hopefully soon new releases of all three will provide the same fixes to those of you who don’t use Gentoo for installing the gems (like, say, if you’re using Heroku to host your application). On the other hand, if you’re a Gentoo user and want to gloat about it, you can tell Rails developer working with OS X that if the Ruby libraries aren’t totally broken is also because we thoroughly test them, more than upstream does!

This is a rant. This is not meant to be constructive, so it is not. If Rich is reading this, he might find some points he can work on; if I had the time, I would be working on them myself; if you feel like you agree with my rant and would like to get the stuff implemented, you can hire me to work on this. But for the rest, this is a rant, so if you’re not in the mood to read my rants, you might want to skip over this.

The laptop from hell is finally shaping up; the smartcard reader works, after editing the ccid files (yes I have to publish the patches up there). Thanks to this I finally wanted to take one further step to make use of it to augment my productivity. One of these things, which I couldn’t feasibly do with OS X, is handling my virtual machines park via virt-manager.

Now, libvirt is designed to be a client-server system, and the server might not be local. There are three transport options to use remote servers: clear-text, TLS, and SSH tunnelling. Now, the clear-text is an obvious bad choice; SSH would be a good choice, if it wasn’t that.. it only works with the root user. There is no way to configure which user to use for the ssh tunnel, and I really don’t want to enable SSH root logins.

TLS is an interesting choice. With this transport, the authentication on the server side is done similarly to what OpenVPN does: you have a personal certification authority (CA), one key/certificate pair for the server, and then one pair per client. All the certificates are signed by the CA itself, and that validates the client to connect to the server. It’s a very nice approach for hosting providers I guess, since you can have a number of workstations that have the certificates set up properly to connect to the farm of virtualisation servers. Unfortunately it has design flawswhen you want to use it with something like a laptop.

First of all, while the server’s certificates and key files’ paths are configurable in the libvirtd.conf file, the client’s files are not configurable, they are hardcoded at build-time based on the system configuration directory (for Gentoo, that’s /etc). They are also only used host-global, as it does not even check for an override in the user’s directory. And this is a double-problem because the certificate has to be passwordless! Or, to put it in a different way, it has to be insecure, lacking a password protection. And since I just said that it does not allow for per-user overrides, you cannot even rely purely on encrypting your home directory. Alternative option is to symlink them from the /etc paths to your home directory but that’s not elegant at all.

It gets even a little worse: to access the display of the virtual machines libvirt tries to do a smart thing, by using the VNC protocol rather than reinventing the wheel. Now, a lot of comments can be written regarding the choice of using the VNC protocol itself, but the fact that it doesn’t reinvent a new one is positive. Unfortunately, when accessing a remote server via TLS, instead of muxing the VNC protocol over the same connection, it simply tries to connect to the VNC display on the remote host. And of course it’s a separate configuration to tell qemu to open the VNC on the correct host. D’oh!

Okay so I get to configure qemu to open the VNC on all interfaces, all IPs… but here’s the catch: for TLS to work correctly, you need to provide correct hostnames, stable hostnames; to do so I decided to use IPv6, since my boxes’ IPv6 are already forward-confirmed, thanks to the latest Hurricane Electric service (providing name server hosting without asking me to maintain my own). Unfortunately, it seems just like qemu does not support IPv6 at all, which means that .. the connection will not work because the hostname will find no hit.

It really doesn’t look too difficult to implement at least part of these features, like choosing the certificate files path, or connecting as a different SSH user. Sure, if you were to shoot high, you could probably consider using a single SCTP socket with multiple channels to multiplex the libvirt protocol together with the VNC connections, but that’s not needed at all. It really just need a few touches here and there to make it much more usable.

Mein Blog startet hier neu, allerdings ausschließlich mit meinen privaten Beiträgen. Alles dienstliche bleibt hier außen vor.

Mein Import hat Probleme mit dem Zeichensatz, sorry für die vielen Sonderzeichen in den alten Beiträgen.

Aber jetzt: los geht’s wieder

I have noted in my post about debug information that in feng I’m using a debug codepath to help me reduce false positives in valgrind. When I wrote that I looked up an older post, that promised to explain but never explained. An year afterwards, I guess it’s time for me to explain, and later possibly document this on the lscube documentation that I’ve been trying to maintain to document the whole architecture.

The problem: valgrind is an important piece of equipment in the toolbox of a software developer; but as any other tool, it’s also a tool; in the sense that it’ll blindly report the facts, without caring about the intentions the person writing the code had at the time. Forgetting this leads to situation like Debian SA 1571 (the OpenSSL debacle), where an “unitialised value” warning was intended like the wrong thing, where it was actually pretty much intended. At any rate, the problem here is slightly different of course.

One of the most important reasons to be using valgrind is to find memory leak: memory areas that are allocated but never freed properly. This kind of errors can make software either unreliable or unusable in production, thus testing for memory leak for most seasoned developers is among the primary concerns. Unfortunately, as I said, valgrind doesn’t understand the intentions of the developers, and in this context, it cannot discern between memory that leaks (or rather, that is “still reachable” when the program terminates) and memory that is being used until the program stops. Indeed, since the kernel will free all the memory allocated to the process when it ends, it’s a common doing to simply leave it to the kernel to deallocate those structures that are important until the end of the program, such as configuration structures.

But the problem with leaving these structures around is that you either have to ignore the “still reachable” values (which might actually show some real leaks), or you receive a number of false positive introduced by this practice. To remove the false positive, is not too uncommon to free the remaining memory areas before exiting, something like this:

extern myconf *conf;

int main() {
  conf = loadmyconf();
  process();
  freemyconf(conf);
}

The problem with having code written this way is that even just the calls to free up the resources will cause some overhead, and especially for small fire-and-forget programs, those simple calls can become a nuisance. Depending on the kind of data structures to free, they can actually take quite a bit of time to orderly unwind it. A common alternative solution is to guard the resource-free calls with a debug conditional, of the kind I have written in the other post. Such a solution usually ends up being #ifndef NDEBUG, so that the same macro can get rid of the assertions and the resource-free calls.

This works out quite decently when you have a simple, top-down straight software, but it doesn’t work so well when you have a complex (or chaotic as you prefer) architecture like feng does. In feng, we have a number of structures that are only used by a range of functions, which are themselves constrained within a translation unit. They are, naturally, variables that you’d consider static to the unit (or even static to the function, from time to time, but that’s just a matter of visibility to the compiler, function or unit static does not change a thing). Unfortunately, to make them static to the unit you need an externally-visible function to properly free them up. While that is not excessively bad, it’s still going to require quite a bit of work to jump between the units, just to get some cleaner debug information.

My solution in feng is something I find much cleaner, even though I know some people might well disagree with me. To perform the orderly cleanup of the remaining data structures, rather than having uninit or finalize functions called at the end of main() (which will then require me to properly handle errors in sub-procedures so that they would end up calling the finalisation from main()!), I rely on the presence of the destructor attribute in the compiler. Actually, I check if the compiler supports this not-too-uncommon feature with autoconf, and if it does, and the user required a debug build, I enable the “cleanup destructors”.

Cleanup destructors are simple unit-static functions that are declared with the destructor attribute; the compiler will set them up to be called as part of the _fini code, when the process is cleared up, and that includes both orderly return from main() and exit() or abort(), which is just what I was looking for. Since the function is already within the translation unit, the variables don’t even need to be exported (and that helps the compiler, especially for the case when they are only used within a single function, or at least I sure hope so).

In one case the #ifdef conditional actually switches a variable from being stack-based to be static on the unit (which changes quite a bit the final source code of the project), since the reference to the head of the array for the listening socket is only needed when iterating through them to set them up, or when freeing them; if we don’t free them (non-debug build) we don’t even need to save it.

Anyway, where is the code? Here it is:

dnl for configure.ac

CC_ATTRIBUTE_DESTRUCTOR

AH_BOTTOM([#if !defined(NDEBUG) && defined(SUPPORT_ATTRIBUTE_DESTRUCTOR)
           # define CLEANUP_DESTRUCTOR __attribute__((__destructor__))
           #endif
          ])

(the CC_ATTRIBUTE_DESTRUCTOR macro is part of my personal series of additional macros to check compiler features, including attributes and flags).

And one example of code:

#ifdef CLEANUP_DESTRUCTOR
static void CLEANUP_DESTRUCTOR accesslog_uninit()
{
    size_t i;

    if ( feng_srv.config_storage )
        for(i = 0; i < feng_srv.config_context->used; i++)
            if ( !feng_srv.config_storage[i].access_log_syslog &&
                 feng_srv.config_storage[i].access_log_fp != NULL )
                fclose(feng_srv.config_storage[i].access_log_fp);
}
#endif

You can find the rest of the code over to the LScube git server — have fun!

Anything I need to know I can find on a website that I can access from almost anywhere. I can't say "I don't know" anymore. I can only say, "I don't know YET."

(What is your biggest 'We are living in the future moment'?)