With the release of Plone 4.1 we have a completely new Windows installer with a much more simple build process which will hopefully lead to more timely releases for Windows installer users. Unfortunately along the way we lost the Plone Controller GUI for starting/stopping instances. Quite independently, Apple's dropping of support for wxPython with the release of Mac OS X Lion means that we're also losing the GUI controller for that platform.

During the 'hallway track' of the Plone Conference I spent a little time discussing this with Alex Limi and Steve McMahon. We all thought the GUI controllers could be really helpful for those downloading and evaluating Plone, but we obviously needed a more sustainable base on which to build a controller. Given we're all web developers, HTML/CSS/Javascript seems the obvious GUI framework. But the prospect of running a special web server to run this makes me uncomfortable, especially considering some people are likely to use the official installer as the basis for production deployments.

Surely then it must be possible to deploy a web application in a dedicated browser window without running a separate web server? Over the past few days I've been investigating the possibilities. Ideally we want a cross platform solution (especially important given the few core developers running Windows.) And a development model that makes both writing the controller and GUI familiar to Python web developers and makes building the Controller application as simple as possible.

Appcelerator Titanium Desktop

Titanium Desktop is a free and Open Source framework for building applications that run in a dedicated WebKit window using HTML/CSS/Javascript with support code running behind it in Python/PHP/Ruby. Appcelerator will build packages of your application for multiple platforms (Mac/Windows/Linux) on their servers. Their model is based on building an AJAX type application with all updates happening over Javascript (or indeed Python) calls that bridge over to the backend code. This makes it difficult to reuse the Request/Response model most web application developers are used to.

Basic app bundle size: 52MB (includes WebKit, but not Python framework)

PyObjC

On the Mac, PyObjC allows Python users to build applications using a variety of APIs exposed by the operating system, including the WebKit framework. One can simply create a window containing a browser control and plug in a custom NSURLProtocol handler to handle requests. There's some simple example code that renders pydocs.

It's easy to see how this model can be extended to run requests through an embedded WSGI app, giving us a familiar development model. Creating a binary with py2app was easy, but clearly small browser part would be Mac only.

Basic app bundle size: 17MB (includes Python, but not WebKit framework)

Qt and PySide

Qt also wraps WebKit and PySide offers an equivalent wrapping to PyObjC. Looking at a couple of examples (one, two) I could quickly see how a very similar application could be built. Qt is cross platform so the same code would run on both Windows and Mac (and Linux as well if required.) Building a working .app bundle is definitely rather tricky. I had some problems running py2app, needing the dev version of it and all its dependencies. Even then it crashed on exit. The alternatives of cx_Freeze, bbfreeze and bundlebuilder also failed in one way or other for me.

Basic app bundle size: 123MB (includes Python, Qt and QtWebKit frameworks)

A tool named Hatchet may be able to help cut this down.

Various other alternatives

From the Python Wiki:

• XULRunner + PyXPCOM. As Python is subordinate in this configuration it would require learning a lot of XUL and XPCOM to get a working application.
• PyWebKitGTK. Not a good solution for the Mac as GTK runs under X11.
• PyWin32 and MSHTML. Would result in a nice small executable for windows users, though on the downside it would involve Win32 programming and using the IE layout engine. I was unable to find much documentation on writing IInternetProtocol handlers with PyWin32.

Conclusion

I think building something with PySide and Qt will be a good first step. It should be simple enough to provide alternative frontends should that be required. Maybe we even want to separate the backend application from the frontend to avoid requiring developers to rebuild the frontend while changing the controller.

Download the example theme from the Diazo theming tutorial I presented at Plone Conference 2011, San Francisco. I'll post a link to the video once it's up.

Now that everyone and their granny is on Facebook, it's no surprise to see the emergence of enterprise social software, with Yammer the most well known example. Starting out as "an enterprise version of Twitter," Yammer is rapidly growing into a fully fledged enterprise social network in the mode of Facebook. Enterprise social software is clearly no fad, we're now seeing this sort of functionality as requirements in RFPs.

Current enterprise social media tools have a major drawback: they are separate from and barely integrated with the main online tool for collaboration and information dissemination within an organisation, the corporate intranet/extranet. That's why I was so excited by the opportunity to talk with Yiorgis Gozadinos of Jarn at the recent Living Statues Sprint about the collaboration tools for Plone he's been building on top of XMPP.

The XMPP standard grew out of Jabber instant messaging project. As such it encompasses messaging, presence information and roster management. It powers Google Talk and Facebook Chat. And crucially it is able to function across HTTP, so browser based clients are available. This makes it an excellent abstraction for building real-time web applications.

At this point I expect those with long experience of Zope may have some concerns... As a threaded application server Zope is usually configured with a small number of fat threads, an architecture completely unsuitable or asynchronous, long-polling applications. Fortunately Yiorgis has an elegant solution: make the Zope server itself an XMPP client and simply offload the asynchronous long-polling requests to ejabberd.

XMPP is not just for chat. The Publish-Subscribe protocol extension (PubSub) enables a broad range of applications to be built on top of XMPP. We believe that the most compelling of these for Plone will be activity streams that could combine user status updates with notifications of activity on the content that concerns you.

Yiorgis' software currently supports microblogging, messaging, and collaborative editing (just like Google Docs!) Check out this screen cast recorded a few days ago:

Collaborative editing with Plone from Yiorgis Gozadinos on Vimeo.

Though it has a different focus, with PubSub XMPP has a lot of overlap with AMQP (RabbitMQ/ActiveMQ). It would certainly be possible to use XMPP for asynchronously executing expensive tasks such as text indexing, content conversion or multi-publishing. And as it spans all the way to the browser, users can be notified when tasks are complete.

I'm really excited about the possibilities that are opening up here. The software's available from github: jarn.xmpp.core. Please do take a look, I know Yiorgis is keen for more people to get involved.

This is just a wee tip, but several people have asked for it so a page would be useful.

Ack is already been discovered by quite a few people, as the URL says it's better than grep. Mostly because it does the right thing in terms of searching directories (the default is to search recursively from the current directory), and the listing formats are much clearer. If you haven't already, get it via. "aptitude install ack-grep".

Configuring to page usefully by default

If you've used ack even once or twice, you'd have noticed that doing "ack | less" is somewhat disappointing. The colour goes away and you're faced with grep style formatting, rather than the colourful ack output you were hoping for. This can be fixed by this one liner alias, which could be placed in your .bashrc.

 alias ack='ACK_PAGER_COLOR="less -x4SRFX" /usr/bin/ack-grep -a'

This does several things to make using ack more pleasant.

• Debian/Ubuntu can't call the executable ack, since something else beat it to it.
• If using ack seriously, at some point you will find out it only searches file extensions it knows about (i.e. not .zcml), and loose several clumps of hairs. -a slows ack down slightly, but lets you keep those hairs.
• The ACK_PAGER_COLOR environment variable means ack output gets paged, but is still colourful. The result of the less options means you get paged results automatically when results fill your screen, without you having to Ctrl-C and re-run through less or something.

Breaking down the options on less:-

• -x4 tells less to use 4 character tabs
• -S tells less to not fold long lines, but let them scroll off to the right.
• -R tells less to use control characters that will colour terminal output, rather than escaping them (used to be -r, thanks Dan Thomas).
• -F tells less to quit if there is less than one screen, so you don't have to decide whether to page or not beforehand.
• -X tells less to ignore an initialization termcap might tell it to do. This is useful for xterms, as they are often set up to send applications like less to a secondary screen, and then switch back afterwards. Without it, -F will cause results to appear on the screen and immediately disappear as less quits.

That's about it. Hope it's useful.

Background

We were tasked with creating a corporate site for Lime Pictures, a media company based in Liverpool. They had already worked on their design internally, and come up with something very stylized, and quite unlike Plone out of the box.

As several other blog posts have already shown, transforming Plone page into something that looks entirely different is quite easy. With this site though, it behaves quite differently too. The navigation is based around a grid, each row being a different section of the site. The first row is special, and shows pages that aren't part of any particular section. This table gives an idea of where each cell in the grid goes.

/

/about-us

/contact

/hollyoaks

/hollyoaks/hollyoaks-later

/hollyoaks/hollyoaks-music-show

/conker-media

/conker-media/the-well

/conker-media/shelfstackers

As most websites start, HTML mockups were created of the various page types. We then defined some Dexterity types that had all the fields required to populate the pages, and Diazo rules to turn these into CMS powered pages. Whilst we can theme the Plone navigation to look different, there's no navigation that behaves like the above.

Weaving in custom navigation with Diazo

Whilst custom navigation can be included using portlets, registering a portlet for each page to get our navigation isn't necessary with Diazo. Normally when you theme with Diazo, you work against the HTML from Plone. However, the "href" attribute available on most directives allows us to reference any URL and include segments from that as well. For example:-

 <append css:theme="div.grid-background"
         css:content="div.grid-background div.footer"
         href="@@menu_view" />

This calls @@menu_view for the current page, and gets the contents of the footer from that. We just need to register menu_view in our product's configure.zcml and we're away. You will notice that Diazo is just selecting div.footer from @@menu_view, this is particularly handy as:-

1. We can make this view generate several bits of useful information for us, not just the footer. plone.app.theming will only fetch @@menu_view once so combining our custom logic into one view makes good sense.
2. Content not selected by the theme is never visible, so is ripe for stuffing with debug when you visit the view directly. For example, CSS classes can be pulled out with javascript and displayed below the elements.

plone.app.contentlisting

However, in throwing away existing navigation portlets we've exposed us to a lot of Python/ZPT to potentially write. There's lots of easy mistakes to make like making private content still visible by using contentValues() too. Fortunately plone.app.contentlisting takes a lot of sting out of this. This is a set of helpers to deal with the lists of objects in templates, and gives a common interface to objects regardless of if they are catalog brains or real objects.

Back to our site. @@menu_view has functions in Python to get lists of notes which the template will simply iterate through and format HTML for each. For example, the function to generate the left hand column is:-

     def getSections(self):
        """Return all sections (top level objects that go in the left-hand column)"""
        sections = [n.getObject() for n
                                   in self.navRoot.restrictedTraverse('@@folderListing')()
                                   if getattr(n,'is_special','')=='menu' and n.isVisibleInNav()]
        sections.insert(0,self.navRoot) # Navigation root is the first item
        return sections

Security is dealt with for me by plone.app.contentlisting, and isVisibleInNav() ensures that the view honours common Plone options from hiding from navigation.

Separating the logic of what is shown where to how it is shown makes for robust automated tests, which if you have a content profile to import as discussed earlier, is really easy.