The core feature was to watch webpages for updates: back then, I did not know about syndication feeds, and I was sick of refreshing ifolder.com every single day hoping for a release of the peer-to-peer version of iFolder (for which we still have no equivalent today).

But why stop there? Why not handle events such as new emails, network failures (in those days, I ran my web server on my flaky residential connexion), software updates becoming available, and so on? You can grasp the whole genius of the idea by looking at one of the first mockups I drew back then:

Some old-timers among you may have noticed a ressemblance with Beagle‘s old user interface — back when it was called “Bleeding-Edge Search Tool”. Aaah, another nostalgic memory of the Novell desktop days.

Specto taught me that even if you have an arguably brilliant concept/mockup, random strangers on the Internet won’t magically jump onto your bandwagon and write code for you. Once this “reality check” finally set in after a couple of months, I took the dive and taught myself Python programming. Specto holds some sentimental value as it was my first OOP learning platform, but I’d probably be horrified if I were to look at its code now, after having considerably improved my skills by contributing to Pitivi. How the hell can Specto have nearly as much code as the current Pitivi development version, anyway!?

But I disgress. Where was I? Ah yes.

GNOME 3.6 made it all obsolete — finally.

The thought struck me recently: the user experience provided by GNOME Shell’s notification system is exactly what I needed all along. It fills my desire to have my computer patiently wait — like a tireless companion — for my return, without worrying about missing events.

If someone had shown me GNOME 3.6 in 2005, I would not have believed it. And yet, with the gradual changes in the 3.x series, we sometimes forget how fantastic the whole thing is.

Even with my heightened expectations, I was still impressed to see that I could now pick up calls even if my home desktop computer’s screen is locked! A real technological delight.

For watching the Web, Specto is now mostly irrelevant: Liferea is the program that took its place in my life. Arguably, social networks have also filled that role for many people.

Sometimes, the software you created becomes irrelevant and you find that to be a good thing.

Following up on my previous announcements where I called for a new maintainer and subsequently decided I wouldn’t touch anything without a patch, I can now say this in light of GNOME 3.6: Specto is dead. I don’t care about it anymore. Of course, if someone wants to take over maintainership/direction of the project, I’ll gladly give that ability to you.

If there are still people out there using this piece of software, go grab the new tarball. Distro maintainers, please package it. This release drops the python glade and gnome dependencies and fixes compatibility with NetworkManager 0.9. See the ChangeLog file for further details.

Thanks to Francesco Marella for some great contributions (gtkbuilder port and code cleanup) that made this possible.

Given that nobody came up wishing to take my place as the maintainer since my previous announcement, I decided on the following new policies (at least until a new maintainer comes up):

• No particular project direction? Screw this. I’ll be deciding unilaterally to do stupid things, like release 0.4 right now (with some amount of smoke testing, of course). In any case, 0.4 can’t be any worse than 0.3.1.
• Anything older than the current release is unsupported. Any old bugs I can’t reproduce will be NEEDINFO’d or closed as obsolete.
• Want new features or any major development? “Patch or it didn’t happen“. I spend most of my time hacking on Pitivi these days and can’t afford to push Specto forward unless someone comes in to help.
• Want to get involved more seriously than a few patches here and there? Feel free to contact me! There’s a ton of work to do.

It is the single most infuriating bug I know in Specto, annoying me daily.

Today, I spent three miserable hours trying to fix it, and went 90% of the way towards that goal. It now correctly parses weird encoding in subject or sender lines in POP3 and IMAP:

…Except that it doesn’t work with GMail IMAP.

Argh.

I’m tired of this crap. The branch is here. Patches welcome.

• I consider Specto “good enough” and I lack the time/hacking skills to devote to it
• I spend lots of energy on other projects such as PiTiVi.

I think by doing so (not maintaining the project as I should be), I am doing a disservice to our users. Thus, it is better to look for another maintainer. This is where you, dear reader(s), come in. Would you like to be owner(s) / maintainer(s) / developer(s) for this project? I spoke with Wout Clymans (my partner in crime for Specto), and he’s in the same situation as me.

Of course, you would not be left alone in the cold with this. New maintainers/developers will be offered guidance to get comfortable with the project and advice (as desired) regarding the direction/evolutionary path it can take (or I could still hang around indefinitely as an adviser/manager if requested).

I just want to see this little baby continue to grow nicely. If it means I lose some control over its direction, then so be it.

What does the future look like? What kind of tasks do we have to accomplish?

Here is a little list of things, off the top of my head:

• Review/commit some patches that have been sitting around
• Redesign the main window and infopane; I have some specific ideas to make it more elegant and efficient
• Fix the bugs listed in the bug tracker
• Upgrade the bzr branch to a more recent format (2a) (shouldn’t be hard, just need to make sure it worked properly)
• Preparing for GNOME 3
  • GNOME Shell integration
  • Port to GTK+ 3.0 when the relevant PyGTK versions come out
  • Migrate from Glade to GTKBuilder Done!
  • Etc.
• Implement a test suite
• Implement new features

One particularly annoying bug for French users was caused by a wrong string in the translation, which caused some watches to be “stuck” and not show notification balloons. See the release notes for more details on what has changed in this minor release.

A tarball is available, and I will try getting someone to build Ubuntu packages (if anyone wants to volunteer to maintain it, you’re welcome to do so), and, if it’s not too late, get it into Ubuntu 9.10.

As you may have noticed, I have made some subtle changes to the actual contents of the notifications. The notification summary is now a proper summary with the watch name instead of “Specto”, and this means we put less “redundant stuff” in the actual “long description”.

Overall, this whole “conform to notify-osd” issue had a positive side-effect: it kicked my ass into actually doing the content cleanup I had on the back of my mind for quite some time.

When the notifications are not attached to the tray icon anymore, messages like this one become glaringly wrong: their uselessness becomes quite apparent. I therefore filed bug 221.

All these changes are currently available for testing in my personal Specto branch. Translators will need to update some strings (mainly just remove obsolete stuff), so please send your updated translations for them to be included in Specto 0.3 final.

Specto is a free and open-source desktop application that will watch configurable events (such as website updates, emails, file and folder changes, system processes, bazaar branches, etc.) and then trigger notifications.

This is a “release candidate” for the 0.3 release. It is considered stable and feature-complete and is intended for last minute testing and to allow you to send us your translations. If you are proefficient in your native language, please help by translating Specto into your language. It is fun, easy, and you get your name in the translator credits. Instructions are available on this page.

Upgrading to this release is strongly recommended, as it solves a huge amount of issues reported in the 0.2.x series, is more stable, implements new features, is more secure and easier to maintain.

This release contains lots of architectural changes. The watch system is now a plugin system, the watch list has changed, GNOME keyring is now used for encrypting passwords by default (though you can disable it), and the configuration/cache directories are now based on the XDG user directory specification. As such, you will need to import your watch list if upgrading from Specto 0.2.x.

Updated packages for Ubuntu are already available on our download page, and might also appear in Ubuntu’s universe in time for 9.04. If you want to provide packages for your favorite distribution, feel free to contact us.

What is new in this release: see the release notes

We are a volunteer project, free “as in beer” and free “as in GNU GPL v2”, and contributions are very welcome. If you like this software and want to fix annoyances that itch you, we value all the help we can get. You don’t need to be a hacking guru! [see contributing page].

Le COMiC et GULUM prévoient faire un installfest Linux le 17 janvier à l’école Polytechnique (Pavillon Lassonde, local M-2101). Vous pouvez aller vous inscrire dans la liste des participants ou installateurs.

Si le public le veut, je donnerai deux présentations au cours de l’événement, traitant de:

• GNOME Power Manager: pourquoi c’est bien pour toi
• Specto: tour d’horizon, historique et découvertes en matière de gestion d’un projet de logiciel libre

Si jamais y’a d’autres suggestions…

What happened there? Well, for making sure that we are using the same “internal” watch list as the actual watches in ~/.config/specto/watches.list (if I’m not mistaken), Specto loaded the file each time and parsed it. The problem is that this is useful (not even 100% sure about it) only when you manually ask Specto to refresh, not during startup, where the watch list is obviously up to date. What Wout did to fix the problem is create a special parameter that makes those method avoid opening and parsing the watch list all the time on startup.

The question remains, however, if the startup time can be further optimized (with the profiling technique in my previous post), or if 2 seconds really is the bottomline; my understanding of profiling outputs is not deep enough to answer that, so any input would be welcome.

Being able to figure out why Specto takes 21 Mib of RAM would also be a nice thing. If it’s Python + GTK’s fault, well, “sho ga nai, desu yo~”. If we can optimize something in our existing code, however, that would be a problem that we should try to address.

I talked about this problem with Wout when I first found out about it, and looked at various profiling tools (sysprof, oprofile, valgrind…), but could not find anything that was supposed to work with Python directly, let alone anything a mortal could use. Thus, we shrugged, and the problem sat there without further investigation, lacking the tools (and time) for that.

Tonight, I wanted to be the next Federico Mena Quintero, and so I tried it again. My first hypothesis was that Specto surely must be doing way too much work, or doing inefficient I/O in some way. I consider it unacceptable that a simple application like Specto does not launch instantly, instead taking as much time as a warm-start OpenOffice.

If you don’t know about software performance profiling, there is some truly awesome “hecho in Federico” material that you must see first. It is easy to understand and funny as hell:

• http://ftp.belnet.be/mirrors/FOSDEM/2007/FOSDEM2007-ProfilingDesktopApplication.ogg
• http://www.gnome.org/~federico/docs/2005-GNOME-Summit/html/index.html
• http://www.gnome.org/~federico/docs/2006-GUADEC/how-much-faster/index.html

So, tonight, I tried Googling for once, and immediately discovered that Python 2.5 already has three official profiling modules! Aided by the quickstart guide, I found to my great delight that I could simply run cprofile over the entire application (since I’m only interested in the startup anyway), instead of butchering the code and wasting tons of time on getting it working.

The normal command that launches Specto if you use “launch.sh” is like this:

PYTHONPATH=$PYTHONPATH:src ./specto $@

The command I used to do the profiling is this, instead:

PYTHONPATH=$PYTHONPATH:src python -m cProfile -s 'cumulative' specto $@ >profile-cumulative.txt

This launches Specto, and when it finally appears and starts refreshing its watches, I immediately hit the “Stop” button and close Specto with a keyboard shortcut, to profile just the bare minimum I need.

I repeated this command with “cumulative”, “time” and “calls” to change the sorting order of the results (other sorting orders are possible).

The results confirmed (it seems) my hunch: iniparser (the module that reads/writes the watch list) is likely the bottleneck: out of the 10 seconds of run time (running the profiler slows things down a bit), about 7 seconds seems to be eaten by iniparser (cumulative time). Now, I’m no profiling guru, so bear with my sloppy analysis, I may be completely wrong here, but I will highlight the parts that seem to me like they are problematic:

Look at all that crap! It must be doing some serious sh** to be able to run over 213 thousand function calls and hog the CPU (a Pentium 4, 2.4 GHz) for over 7 seconds…

Remember Federico’s law? Trim the fat in this order (after having asked the profiler):

1. Excess work
2. Excess I/O
3. Bad algorithms
4. Bad memory access patterns
5. Micro-optimizations (function call overhead, mis-predicted branches, CPU-level foo)

… and it seems like Specto’s iniparser module is doing a lot of “excess work”. At the same time, why is “time.sleep” eating 1.6 seconds of our startup time? If I recall correctly, we don’t actually use a sleeping timer during startup before the watches start refreshing. This could be another path of investigation.

Now what? Throw iniparser away? I guess not, not until someone sits down and actually investigates (better than what I did here):

1. What to blame: our code, or iniparser? We should look in Specto’s code, to see if there is some dumb/flawed logic, like “open and read the file each time for every watch”.
2. If iniparser is at fault, can it be optimized, or must it be replaced by some other data storage method?

So, this concludes my initial blog post on profiling Specto. I have here identified some potential performance bottlenecks, and I would appreciate any help or feedback on analyzing this issue more accurately. If someone wants to actually tackle the problem, you’re more than welcome. I personally don’t have time, at least not until Christmas holidays, and even then, I can’t promise anything.