How to think in Plone (when those  about you think in Drupal)

Dan Jacka

• Let’s make switching to Plone easier
• Plone content is naturally structured, drupal content has nodes indentified by numbers.
• For Plone lots of content is a prerequisite
• Confusions when coming from Drupal; content ‘lives’ where it appears and restrict locals adds with permission
• ‘micro components’ are powerful ways to add functionality
• Use micro components like dexterity behaviours, collective.contentleadimage, collective.watcherlist. Small products are good!
• Big products are bad, if you only need to use a part and get unnecessary stuff you don’t need.
• With Zope Component Architecture you can override specific parts in Plone, like the breadcrumbs 
• Store data; With context use zope annotations and archetypes.schemaextender. Without context use plone.app.registry, portal_properties and local utilities
• See:  http://www.martinaspeli.net/articles/dcworkflows-hidden-gems

ZODB tips and tricks

Carlos de la Guardia

• See collective.zodbbrowser Provides access to all objects and their attributes
• Eye is an external tool to browse the database, without having to install (in buildout)
• OMG POSKey error: first backup! Fire up debugger, see  http://plonechix.blogspot.com/2009/12/definitive-guide-to-poskeyerror.html
• Get rid of persistent utilities, see wildcard.fixpersistentutilities
• Always backup first before trying to fix anything!
• Restore data / do an undo from data that is deleted a long time ago. Use zc.beforestorage, a wrapper around storage to show site like it was on a certain date
• RelStorage, drop in replacement for file storage. Designed for high volume sites, multiple zodb instances can share the same database. Starts quickly regardless of db size. Supports undo, packing blobs. Capable of fail-over to replicated databases.
• Zc.zodbactivitylog (track db actvity), zodbshootout (benchmarks zeo vs relstorage), zodbupdate (helps you rename classes), dm.historical (get history of objects), dm.zodb.repair (restore lost objects)
• See Products.ZMIntrospection to look in objects
• Use PersistentDict for small amount of items, use OOBTree (and friends) for large amounts
• Use BTrees.length instead of len, much faster and avoids conflict errors
• Use zc.zlibstorage for dbs with lots of text, saves 60/70% of storage
• Use zc.zodbgc an inverse graph of db
• Increase the ZEO client cache size, when everything is crumbeling around you. When increasing round trips to ZODB are increased   

Carlos is writing a book on the zodb: http://zodb.readthedocs.org/en/latest/index.html 

Mistakes made and lessons learnt

Matt Hammilton And Matt Sital-Singh

• Use case a e-learning system, lot of content, users editing content but not necessary aware of it, lots of clustering of load on resources
•  30.000 user accounts, 160.000 messages, QA objects etc
• Don’t create  ghost content. If you need shared content, it can be a good idea to don’t use subsites but sync the content
• Use optimization products. See experimental namespace; queryplan, contentcreation, daterangeindexoptimisations, aggresiveopaquesetup, indexing. Most of them aren’t experimental anymore and are included in plone 4.2
• Lot’s of answers/questions in catalog. Up to 1.400.000 qa objects. Uhm sql? Or catalog multiplex tool, separate these objects from main catalog.
• Pre-load Catalog QueryPlan
• Keep things you don’t need out of the catalog
• If you know where something is – Don’t search for it. Hardcoding is ok
• Use unrestrictedSearchResults if possible, the catalog has a lot less work to do
• Maybe should not have used Plone. A lot of the data is relational.

Beginner mistakes. Expert failures

Alan Runyan

Beginners:

• Use logging/sending email or long term persistence for error logs
• Don’t mix dev/stage/prd concept/concerns
• Don’t use directlyProvides or using the ZMI to apply marker interfaces
• Do learn the profiler, see collective.stats (how many load/stores per request)
• Don’t attempt to ‘normalize’ the model
• Willing to claim defeat and backtrack; sometimes you simply do
• When beginning development use an alpha, beta or RC. If the project is finished, chances are big a stable version is released.
• Don’t be willing to take pain. Give feedback!

Experts:

• Using components early, not writing reusable code is ok.
• Overloading too much functionality in view
• Mounting / splitting ZODB in effort to make things faster
• Using Plone inappropriately. Security, workflows, staging/versioning, content, add-on components.
• See book scalability rules
• Unwilling to thorougly understand tradeoffs between ZODB/RDBMS
• Masking over performance problems with caching

Check out ploud and  ptah .

RelStorage is stable and is used in many production sites. It works with PostgreSQL, MySQL and Oracle. The rationale to use it is replication. SQL datbases can do this, the ZODB can’t.

Dextery in the wild

David Glick

• Case study of a complex Dexterity based solution, Net Impact
• Sales Force integration
• Membrane is used to represent users as content items. See dexterity.membrane
• Membrane the ugly: extra catalog with unneeded index
• Using dexterity with grok saves writing ZCML
• Content items are defined in (super) models
• Content items have multiple forms with different fields. Auto form can’t be used, dexterity.EditForm class view is used.
• See collective.z3cform.datagridfield, eea.facetednavigation, collective.salesforce.content  (dexterity behaviour)
• Dexterity content types is only used for custom content. Existing content type are modified using schema extender.

Tsunami proof Plone

Adam Terrey

• Building high available “web scale” Plone platform for emergency services
• Requirements; scale-ability, multisite, 99,99% reliability
• The cloud isn’t reliable enough
• CDNs are used for serving content
• Even the slightest amount of caching let’s Plone ‘breathe’ under heavy load
• Everything in the hosting stack must be redundant. Expensive!
• Datacenter should have multiple backbones
• DB redundancy is done with relstorage
• Funkload is used for stress testing

Progressive enhancement with wsgi 

Matthew Wilkes

• Wsgi is just an API for handeling http requests
• Use wsgi as middleware to add functionality to your application
• Good wsgi libraries: WebOb, makes requests easy to deal with / wsgirefWSGI web server is in the std lib
• The ZopeSkel equivilent for wsgi: wsgitemplates
• Example captcha’s, most form libraries have a different way to implement captcha’s
• Uses wsgi middleware to rewrite the form
• The backend application has a checkbox to see if the user is human. Middleware inserts a captcha widget. If theform is submitted the middleware checks the captcha input and on valid input the ’is human’ box is checked. On valid captcha input the form is stored, else an error is shown.
• Probably overkill, more initial effort but can be easy to apply to customer sites.
• Example for how to use wsgi middleware
• See islay.simplecaptcha and islay.hardercaptcha

Multiplayer Plone : Realtime collaboration 

Geir Bækholt

• Jabber/XMPP protocol is used, realtime and asyn
• Presence (status), message, iq (subscription),
• Communication not directly thru Plone, but via javascript
• Scales to large amount of users
• Collaborative writing like Google Docs in ATContent and dexterity
• Probably same diff match patch algorithm as in google docs
• Future: content notifications, async queues with xmpp in plone, conferencing, video and audio chat.
• See jarn.xmpp.buildout 

Killer feature/selling point for intranets and sites with lots of authors.

Clone to Plone

Adam Terrey

• How to move from older ‘systems’ to Plone. Migrating content, it’s pretty hard.
• Funnelweb http://plone.org/products/funnelweb
• A web crawler, uses mr.migrator and transmogrifyer
• Buildout, command line or Plone TTW

• Crawler is configurable; ignore regex, drop content using tal, amount of items, ignore robots meta
• Crawler saves in cache on disk
• Content extraction using xpath
• Restructure content, tidy titles
• Upload to Plone

• What’s not converted: dynamic content, front pages, collections, portlets.
• Diazo is used to create to look and feel of the old site
• Used to migrate big sites, est time to migrate a big site is 3 / 4 days

Making Plone Mobile using Responsive Web Design

Rob Porter

• Responsive design, geared towards lot’s of devices (and screen sizes!)
• Why design websites for mobile? Because there’s a huge increase on mobile device usage.
• See http://bostonglobe.com http://forefathersgroup.com and make the browser window smaller. The layout changes but content stays the same.
• See http://mediaqueri.es
• Define a range of screen resolutions with @media, to cater different devices. Each resolution range has an own layout/style/css.
• See plonetheme.responsivetheme, makes Plone responsive
• Book to buy Responsive Web design – Ethan Marcotte
• First start with smallest screen sizes and then go bigger

Creating websites using responsive desgn better than creating specific iDevice, Android, Blackberry apps.

The future search of Plone

Sally Kleinfelt, Hanno Schlichting and someone from six feet up

• Information retrieval
• Two search engines ZCatalog and Solr
•  ZCTextIndex is very simple, TextIndexNG adds multilingual, better parsing, binary transforms, synonyms.
• Solr, based on Lucene Java search library. RESTful APIs. Powers twitter and wikipedia. An army of engineers working on it
• Solr has way more search features than ZCatalog
• Collective.solr and alm.solrindex are available add-ons. Collective.recipe.solrinstance as buildout recipe
• Solr indexing is not done transactional, it takes a while before the contt is indexed

De PyGrunn ‘Python and friends’ conferentie is gehouden in Het Paleis in Groningen en is bezocht door ruim 80 personen. Voorafgaand aan de conferentie was er een masterclass van Armin Ronacher. De conferentie is gericht op de Python programmeertaal en verwante technologie. Met sprekers uit Nederland en het buitenland was de Python-community goed vertegenwoordigd op PyGrunn. Vanuit Goldmund, Wyldebeast & Wunderliebe heeft Kim Chee Leong een presentatie gegeven over buildout, dit is een tool voor het uitrollen van applicaties.

Goldmund, Wyldebeast & Wunderliebe en Paylogic hebben de conferentie georganiseerd. Vanwege het grote success zal PyGrunn er ook volgend jaar weer zijn.

This example is applicable to Plone 3 and 4. But the jQuery/Maps API are generic, so if you’re not interested in the Plone stuff just skip to the jQuery code.

Use case:

• Select a location based on an address or by dragging a marker in the map. Store latitude, longitude and zoom level.
• Display a Google Map widget on your content type that show a stored location

Prequisites

• You know how to create your custom content type in Plone (if you want to integrate this example in Plone).
• Basic javascript/jQuery skills

Getting started

Your custom content type

We want to store the longitude, latitude and zoom level in the content.

Create you own custom content types and add the following fields:

• longitude and latitude in a string field
• zoomLevel in a integer field

Include necessary javascript and CSS

We are including jQuery. jQuery UI with only the live search function included, this allows direct results when searching for a location. The Google Maps API v3. And some CSS for the map canvas and the live search field.

If you’re planning to use the jquery libraries from my site, please download the files and host these on your own site.

<script src="http://www.leong.nl/wp-content/jquery-1.4.2.min_.js" type="text/javascript"><!--mce:0--></script>
 
<script src="http://www.leong.nl/wp-content/jquery-ui-1.8.1.custom.min_.js" type="text/javascript"><!--mce:1--></script>
 
<script src="http://maps.google.com/maps/api/js?sensor=false" type="text/javascript"><!--mce:2--></script>
 
<style type="text/css">
#map_canvas {        
  width: 650px;    
  height: 500px;        
  margin-bottom: 2em;  
} 
 
.ui-autocomplete  {        
  background-color: white;       
  width: 300px;        
  border: 1px solid #cfcfcf;        
  list-style-type: none;        
  padding-left: 0px;        
  cursor: pointer; 
}  
</style>

The Google Maps jQuery code

This is jQuery code for displaying and selecting locations in Google Maps. Most of the Maps API code used in this example comes from this article on cibul.org. The article has a good explanation how the Maps API works.

Short explanation on the code below. The inserting of the Maps and search field is all done in javascript code. Without this code you can still save or view the lat/long coordinates. Based on editing or viewing content different functions are called.

 
// based on: http://tech.cibul.org/geocode-with-google-maps-api-v3/
// replace jq with $ character if you're using this code outside Plone
 
var DEFAULT_LATITUDE = 53.21310;
var DEFAULT_LONGITUDE = 5.71713;
var DEFAULT_ZOOM_LEVEL = 9;
 
var ploneMaps = {
 
    /**
     *  Show the map based on saved location
     */
    googleMapsView: function(message) {
        jq('#archetypes-fieldname-latitude').hide();
        jq('#archetypes-fieldname-longitude').hide();
        jq('#archetypes-fieldname-zoomLevel').hide();
 
        var html = "<label>Location:</label><br/>";
        html += "<div id='map_canvas'></div>";
        jq(html).insertBefore('#archetypes-fieldname-latitude');
 
        var longitude = parseFloat(jq("#parent-fieldname-longitude").text());
        var latitude = parseFloat(jq("#parent-fieldname-latitude").text());
        var zoom = parseInt(jq("#parent-fieldname-zoomLevel").text());
 
        ploneMaps.googleMapsDraw(longitude, latitude, zoom, true, false);
    },
    /**
     *  Show the saved location or starting point.
     *  Set-up a search widget for address geocoding
     */
    googleMapsEdit: function(message) {
        jq('#archetypes-fieldname-latitude').hide();
        jq('#archetypes-fieldname-longitude').hide();
        jq('#archetypes-fieldname-zoomLevel').hide();
 
        // create search field
        var html = "<div class='field' id='google-map-search'>";
        html += "<label for='map-search'>Location</label>";
        html += "<div class='formHelp'>Use the marker to select a location ";
        html += "or search field to select a location.</div>";
        html += "<input name='map-search' type='text' size='45'></input>";
        html += "<div id='map_canvas'></div>";
        html += "</div>";
 
        jq(html).insertBefore('#archetypes-fieldname-latitude');
 
        // no form submit when an enter is given in search field
        jq('#google-map-search').keypress(function(e){
           if(e.which == 13){
               e.preventDefault();
           }
        });
 
        // save zoomlevel on submit
        jq('form#plaats-base-edit').submit(function(e){
           jq("#zoomLevel").val(map.getZoom());
        });
 
        // saved location
        var latitude = jq('#latitude').val();
        var longitude = jq('#longitude').val();
        var zoomLevel = parseInt(jq('#zoomLevel').val());
 
        // use default location if nothing is saved
        if (!latitude || !longitude) {
            latitude = DEFAULT_LATITUDE;
            longitude = DEFAULT_LONGITUDE;
            zoomLevel = DEFAULT_ZOOM_LEVEL;
        }
 
        var data = ploneMaps.googleMapsDraw(longitude, latitude, zoomLevel, true, true);
        var map = data[0];
        var markerObj = data[1];
 
        ploneMaps.googleMapsHelper(map, markerObj);
    },
    /**
     * Draw the location based on specific parameters
     */
    googleMapsDraw: function(longitude, latitude, zoomLevel, marker, drag) {
        var map = false;
        if (!isNaN(longitude) && !isNaN(latitude)) {
            var location = new google.maps.LatLng(latitude, longitude);
 
            var myOptions = {
              zoom: zoomLevel,
              center: location,
              mapTypeId: google.maps.MapTypeId.ROADMAP
            };
            map = new google.maps.Map(document.getElementById("map_canvas"), myOptions);
            if (marker) {
                marker  = new google.maps.Marker({position: location, map: map, draggable: drag});
            }
 
        }
        return [map, marker]
    },
    /**
     * Helper function, ajax address search and marker event listener
     */
    googleMapsHelper: function(map, marker) {
        //GEOCODER
        geocoder = new google.maps.Geocoder();
        // Form auto completion
        jq("input[name='map-search']").autocomplete({ 
            //This bit uses the geocoder to fetch address values
            source: function(request, response) {
                geocoder.geocode( {'address': request.term, 'region': 'NL' }, function(results, status) {
                  response(jq.map(results, function(item) {
                    return {
                      label: item.formatted_address,
                      value: item.formatted_address,
                      latitude: item.geometry.location.lat(),
                      longitude: item.geometry.location.lng()
                    }
                  }));
                })
            },
            // This bit is executed upon selection of an address
            select: function(event, ui) {
                jq("#latitude").val(ui.item.latitude);
                jq("#longitude").val(ui.item.longitude);
                var location = new google.maps.LatLng(ui.item.latitude, ui.item.longitude);
                marker.setPosition(location);
                map.setCenter(location);
                map.setZoom(11);
            }
        });
        // Reverse Geocoding, when the marker is being dragged on the map update the fields.
        google.maps.event.addListener(marker, 'drag',  function() {
            geocoder.geocode({'latLng': marker.getPosition()}, function(results, status) {
              if (status == google.maps.GeocoderStatus.OK) {
                if (results[0]) {
                  jq("input[name='map-search']").val(results[0].formatted_address);
                  jq("#latitude").val(marker.getPosition().lat());
                  jq("#longitude").val(marker.getPosition().lng());
                }
              }
            });
        });
    }
}
 
jq(document).ready(function() {
 
    // match this jquery selector on the view template of your content type
    if (jq('body.template-plaats_view.portaltype-plaats').length >= 1) {
        ploneMaps.googleMapsView();
    }
    // match this jquery selector on the edit template of your content type
    else if (jq('body.template-base_edit.portaltype-plaats').length >= 1) {
        ploneMaps.googleMapsEdit();
    }
 
});

This was not a good solution for restoring specific items because the version history was broken/corrupt for all items. Saving an item that was restored the following exception was shown: “ArchivistUnregisteredError: The object …“. When accessing the history this error was shown “AttributeError: ‘list’ object has no attribute ‘getLength’“.

The solution was resetting the version history using the CMFEditions tool. You can reset the version history for an item this way:

hist = getToolByName(context, 'portal_archivist')
prepared =  hist.prepare(obj)
hist.register(prepared)

This solved the ArchivistUnregisteredError for me.

Here are a few tips for hardening your Debian/Ubuntu server.

SSH key based authentication

Only allow logins using public SSH keys. This way we prevent brute force attacks. Create private and public keys using the ssh-keygen command. First copy the public key from your pc to the server using:

$ ssh-copy-id -i .ssh/id_rsa.pub user@host

Test if you can login with your public key. The public key is stored in .ssh/authorized_keys. So if you add a new user ask them their pub key and copy this into authorized keys.

Change /etc/sshd_config to disable password based logins:

ChallengeResponseAuthentication no
PasswordAuthentication no

Filesystem permissions

The default user permission has umask 022 where other/world user also have access. Using umask 007 the owner en group has rw access, other/world hasn’t got any access.

Change default umask 022 to 007:

/etc/profile
/etc/login.defs
/etc/init.d/rc

Mounted volumes must have proper permissions

Add two mount options in /etc/fstab for partitions that have no suid programs and no device nodes. Options are called nosuid and nodev.

Example:

/dev/sda5       /tmp            ext3    defaults,nosuid,nodev        0       2
/dev/sda6       /var            ext3    defaults,nosuid,nodev        0       2
/dev/sda7       /data2          ext3    defaults,nosuid,nodev        0       2

Dedicated group for su

Only allow users in adm group to become root using pam_wheel.

In /etc/pam.d/su uncomment this line and add group part.

auth       required   pam_wheel.so group=adm

Add sysadmins to adm group

usermod -a -G adm [username]

Separate temp directories for users

Using pam tmpdir modules each user has a separate tmp dir. So instead of using /tmp for everyone, each user gets a /tmp/user/USERID directory. A user cannot see the temp files of other users.

Install the tmpdir pam module:

apt-get install libpam-tmpdir

Add the following line to /etc/pam.d/common-session

session    optional     pam_tmpdir.so

Know when security updates are available

Keep the packages on the server up to date. Use the mail* to functionality in crontab to get a automated warning when an update is available. It saves the hassle for checking manually. It’s not recommended to run the updates unattended.

Add this script in /usr/local/bin/check_security.sh

#!/bin/sh
apt-get update -qq -s > /dev/null 2>&1
 
LISTFILE=$TMP/check_security.lock
UPGRADE_CMD="apt-get upgrade -qq -s"
MACHINE=`hostname`
LIST=""
 
for package in `$UPGRADE_CMD | grep -e '^Inst' | awk '{ print $2 }'`;
do
  LIST="$LIST $package"
done
 
if [ -z "$LIST" ]; then
  if [ -e $LISTFILE ]; then
    rm -f $LISTFILE
  fi
  exit 0
else
  if [ ! -e $LISTFILE ]; then
    echo $LIST > $LISTFILE
    echo "Please run security updates on ${MACHINE}!"
    echo "=========================================="
    echo $LIST
  fi
fi

Add the script to crontab for root user.  Enter your e-mail address in the mailto variable. Add the script to crontab. Check each hour is an update is available:

MAILTO='me@domain.com'
5 * * * * /usr/local/bin/check_security.sh

Install the nullmailer and  mailutils package to allow mail relaying. Enter the hostname and smtp server in install dialog.

apt-get install nullmailer mailutils

Clean old files/dirs in temp directory

Remove files and directories older than 30 days. The tmp dir is for temporary files!

Add a shell script in /usr/local/bin/clean_tmp.sh

#!/bin/bash
# GW20e - KC
# Clean tmp directory, we don't want files/dirs older
# than 30 days.
find /tmp/* -type f -mtime +30 -exec rm -f {} \;
find /tmp/* -type d -mtime +30 -exec rm -rf {} \;

Add the script in crontab so it’s executed every night:

0  5 * * * /usr/local/bin/clean_tmp.sh > /dev/null

Only a minimal set of network services must be provided

Only run the network services that are needed. Each service can bring in a security risk. Configure the network services so they only listen on specific interfaces.
Run netstat to check which service is listening on what interface. Example:

server:~# netstat -nlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 91.194.xxx.xxx:18080    0.0.0.0:*               LISTEN      9544/python2.6
tcp        0      0 91.194.xxx.xxx:5666     0.0.0.0:*               LISTEN      3515/nrpe
tcp        0      0 127.0.0.1:111           0.0.0.0:*               LISTEN      9035/portmap
tcp        0      0 192.168.3.45:8080       0.0.0.0:*               LISTEN      3876/python2.6
tcp        0      0 192.168.3.45:28080      0.0.0.0:*               LISTEN      3875/python2.6
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      3714/apache2
tcp        0      0 192.168.3.45:8081       0.0.0.0:*               LISTEN      3877/python2.6
tcp        0      0 192.168.3.45:22         0.0.0.0:*               LISTEN      3505/sshd
tcp        0      0 127.0.0.1:761           0.0.0.0:*               LISTEN      3553/famd
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      3714/apache2
udp        0      0 127.0.0.1:111           0.0.0.0:*                           9035/portmap

Disable network services or bind  the service to a specific interface. Normally portmap is listening on all interfaces.

Bind portmap to localhost loopback

In the above example portmap is only listening on the localhost loopback. Bind the portmap service to localhost, edit ”/etc/default/portmap”. Uncomment this line:

OPTIONS="-i 127.0.0.1"

Disable IPv6

If you’re not using IPv6, disable it to prevent possible vulnerabilities. Add the following file /etc/modprobe.d/00local

alias net-pf-10 off
alias ipv6 off

Apache webserver

Don’t allow directory indexes by disabling autoindex module

a2dismod autoindex
/etc/init.d/apache2 restart

Apache has a separte config file for security. Edit /etc/apache2/conf.d/security and change the following settings:

# Don't give away info about OS and compiled in modules
ServerTokens Prod
# Don't show server version in server-generated pages
ServerSignature Off
# Disables HTTP trace, only used for debuging purposes. Potential security vulnerability.
TraceEnable Off

This article has good tips for securing the Apache webserver: MDLog:/sysadmin – Apache Tips & Tricks.

Do a security audit for the system

Lynis is an excellent tool to audit the system. Download the latest tarball, decompress and run. Lynis is an auditing tool which tests and gathers (security) information for *nix based systems.

If you want to be on the safe side, and be sure your server is secure, hire an independent company for a security audit.

Amazon S3 storage

Amazon is offering a wide array of cloud services. The storage service is called Amazon S3 (simple storage services). The storage is really cheap, approximately  $0.15 per GB/month for storage and $0.10 GB/month for data transfer. Dropbox and Ubuntu One are using S3 as storage facility, Twitter and Slideshare are using S3 to host images.

Using S3 as online storage isn’t that hard, it takes a bit more effort than using a service like Dropbox. But you’ll get unlimited storage for a bargain. Amazon isn’t offering a client to mount S3 storage on your computer. It provides an interface for developers, allowing connection to the storage. Luckily others have done the heavy lifting and there is software available to use the S3 storage on your computer. I don’t know the specific clients for Mac or Windows, google is your friend. I’m using FuseOverAmazon (this specific fork for EU buckets) on Ubuntu Linux.

So sign up for Amazon S3 and do some googling which client you need for your operating system. Now I only need a faster internet connection so I don’t have to wait when uploading gigabytes of photo’s to S3!

Together with my collegue Kees Hink we gave a presentation about Hardening Plone.

For a customer who needed a DMS to exchange documents with third parties we hardened the Plone stack. Several highlights of the hardening part are modifcations in the OS,� two technical audits, a process audit and adding some extra products in Plone. Here’s is the recorded stream of the presentation: http://www.ustream.tv/recorded/2446265. Here are the slides:

I use Setuptools 0.6c9 to create Python eggs. It happens that the latest Setuptools won’t play well with subversion 1.6 (shipped with Karmic).

When creating an egg with:

python setup.py bdist_egg

I got this error:

subversion unrecognized .svn/entries format

You can fix this by running a patch from the setuptools team, download the most recent patch from the issue page. Find out where setuptools lives in your site-packages and apply the patch. More details here if you’re not familiar with site-packages and applying patches.

Update 25 May 2010 – This information is outdated. Please refer to this page on the XDA Developers forum:

http://forum.xda-developers.com/showthread.php?t=529062

Here are the steps to the how-to’s to get root access to your HTC Magic and load a custom ROM. I took me some time to find all the needed info to do the job. So I’m hapy to share it. There’s a lot of (cluttered) info but most of it is found in multiple page forums. Really great that there lot’s of people developing and using this stuff but a forum isn’t the right place for documentation.

Note 30-03-2010, the information below is outdated. Please look at the following wiki’s for more up to date info:

http://wiki.xda-developers.com/index.php?pagename=HTC_Sapphire_Hacking

http://wiki.cyanogenmod.com/index.php/Main_Page

Got root?

Getting root access on the Magic isn’t so hard. Just install the SDK and USB drivers and push the images to the phone. In the last step is to install haykuro’s SPL update. This is the bootloader (correct?) and has a very usefull back-up option. It’s called nandroid and creates back-up images of the system. After this you have root access from the android debugger on your computer.

http://android-dls.com/wiki/index.php?title=Magic_Rooting

The second step is enabling root access from the phone. This allows you to su from the phone. Needed for installing rooted apps. It opens a security risk as mentioned in the how-to. But there a sudo-like app SuperUser whitelist to prevent unwanted root access (preinstalled on most custom roms). :

http://android-dls.com/wiki/index.php?title=Magic_Root_Access

Custom ROMS

So now you’ve got full access to the Magic but stuck with a stock rom from your provider. There are lots of roms floating around for Android but most of them are for the HTC Dream (G1). I found out the hard way by installing a G1 rom, WiFi and other hardware functions aren’t working. Not so strange with a different kernel etc…

These are the ROMs that are available at the moment:

• Google/HTC original stock rom
• Hero Rom. Android with an updated ui. Shiney and new but it’s slow and laggy
• Google ION rom. ION is a developer phone. Customized rom available.
• Haykuro’s Rom. Haykuro’s roms are excellent but it seems that he has stopped developing.
• Smartphone France (translation). Customized ION rom, available in EU languages.

Sending a rom is really easy, just like in the rooting process with an update.zip on the SD card. Always take care when installing a new rom. Better safe than sorry and check if it works on your Magic/Sapphire/ION. The last thing you want is a bricked phone…

I tried the Google ION rom and it works well. The benefits of this custom rom are; it’s faster, rooted, voice commands and more!� This is a good rom but there’s a HTC soft keyboard instead of Google’s. I didn’t like it and replaced it. Instructions here, you’ll need a specific rom (for extracting system files) or it won’t work!

I’m now using the Smartphone France version. The main advantage between above ION rom is this one is, tethering works and a higher version update rate. You won’t notice anything of the French language (beside a few small apps on the rom).

Also check out my Android bookmarks!