How to think in Plone (when those  about you think in Drupal)

Dan Jacka

• Let’s make switching to Plone easier
• Plone content is naturally structured, drupal content has nodes indentified by numbers.
• For Plone lots of content is a prerequisite
• Confusions when coming from Drupal; content ‘lives’ where it appears and restrict locals adds with permission
• ‘micro components’ are powerful ways to add functionality
• Use micro components like dexterity behaviours, collective.contentleadimage, collective.watcherlist. Small products are good!
• Big products are bad, if you only need to use a part and get unnecessary stuff you don’t need.
• With Zope Component Architecture you can override specific parts in Plone, like the breadcrumbs 
• Store data; With context use zope annotations and archetypes.schemaextender. Without context use plone.app.registry, portal_properties and local utilities
• See:  http://www.martinaspeli.net/articles/dcworkflows-hidden-gems

ZODB tips and tricks

Carlos de la Guardia

• See collective.zodbbrowser Provides access to all objects and their attributes
• Eye is an external tool to browse the database, without having to install (in buildout)
• OMG POSKey error: first backup! Fire up debugger, see  http://plonechix.blogspot.com/2009/12/definitive-guide-to-poskeyerror.html
• Get rid of persistent utilities, see wildcard.fixpersistentutilities
• Always backup first before trying to fix anything!
• Restore data / do an undo from data that is deleted a long time ago. Use zc.beforestorage, a wrapper around storage to show site like it was on a certain date
• RelStorage, drop in replacement for file storage. Designed for high volume sites, multiple zodb instances can share the same database. Starts quickly regardless of db size. Supports undo, packing blobs. Capable of fail-over to replicated databases.
• Zc.zodbactivitylog (track db actvity), zodbshootout (benchmarks zeo vs relstorage), zodbupdate (helps you rename classes), dm.historical (get history of objects), dm.zodb.repair (restore lost objects)
• See Products.ZMIntrospection to look in objects
• Use PersistentDict for small amount of items, use OOBTree (and friends) for large amounts
• Use BTrees.length instead of len, much faster and avoids conflict errors
• Use zc.zlibstorage for dbs with lots of text, saves 60/70% of storage
• Use zc.zodbgc an inverse graph of db
• Increase the ZEO client cache size, when everything is crumbeling around you. When increasing round trips to ZODB are increased   

Carlos is writing a book on the zodb: http://zodb.readthedocs.org/en/latest/index.html 

Mistakes made and lessons learnt

Matt Hammilton And Matt Sital-Singh

• Use case a e-learning system, lot of content, users editing content but not necessary aware of it, lots of clustering of load on resources
•  30.000 user accounts, 160.000 messages, QA objects etc
• Don’t create  ghost content. If you need shared content, it can be a good idea to don’t use subsites but sync the content
• Use optimization products. See experimental namespace; queryplan, contentcreation, daterangeindexoptimisations, aggresiveopaquesetup, indexing. Most of them aren’t experimental anymore and are included in plone 4.2
• Lot’s of answers/questions in catalog. Up to 1.400.000 qa objects. Uhm sql? Or catalog multiplex tool, separate these objects from main catalog.
• Pre-load Catalog QueryPlan
• Keep things you don’t need out of the catalog
• If you know where something is – Don’t search for it. Hardcoding is ok
• Use unrestrictedSearchResults if possible, the catalog has a lot less work to do
• Maybe should not have used Plone. A lot of the data is relational.

Beginner mistakes. Expert failures

Alan Runyan

Beginners:

• Use logging/sending email or long term persistence for error logs
• Don’t mix dev/stage/prd concept/concerns
• Don’t use directlyProvides or using the ZMI to apply marker interfaces
• Do learn the profiler, see collective.stats (how many load/stores per request)
• Don’t attempt to ‘normalize’ the model
• Willing to claim defeat and backtrack; sometimes you simply do
• When beginning development use an alpha, beta or RC. If the project is finished, chances are big a stable version is released.
• Don’t be willing to take pain. Give feedback!

Experts:

• Using components early, not writing reusable code is ok.
• Overloading too much functionality in view
• Mounting / splitting ZODB in effort to make things faster
• Using Plone inappropriately. Security, workflows, staging/versioning, content, add-on components.
• See book scalability rules
• Unwilling to thorougly understand tradeoffs between ZODB/RDBMS
• Masking over performance problems with caching

Check out ploud and  ptah .

RelStorage is stable and is used in many production sites. It works with PostgreSQL, MySQL and Oracle. The rationale to use it is replication. SQL datbases can do this, the ZODB can’t.

Dextery in the wild

David Glick

• Case study of a complex Dexterity based solution, Net Impact
• Sales Force integration
• Membrane is used to represent users as content items. See dexterity.membrane
• Membrane the ugly: extra catalog with unneeded index
• Using dexterity with grok saves writing ZCML
• Content items are defined in (super) models
• Content items have multiple forms with different fields. Auto form can’t be used, dexterity.EditForm class view is used.
• See collective.z3cform.datagridfield, eea.facetednavigation, collective.salesforce.content  (dexterity behaviour)
• Dexterity content types is only used for custom content. Existing content type are modified using schema extender.

Tsunami proof Plone

Adam Terrey

• Building high available “web scale” Plone platform for emergency services
• Requirements; scale-ability, multisite, 99,99% reliability
• The cloud isn’t reliable enough
• CDNs are used for serving content
• Even the slightest amount of caching let’s Plone ‘breathe’ under heavy load
• Everything in the hosting stack must be redundant. Expensive!
• Datacenter should have multiple backbones
• DB redundancy is done with relstorage
• Funkload is used for stress testing

Progressive enhancement with wsgi 

Matthew Wilkes

• Wsgi is just an API for handeling http requests
• Use wsgi as middleware to add functionality to your application
• Good wsgi libraries: WebOb, makes requests easy to deal with / wsgirefWSGI web server is in the std lib
• The ZopeSkel equivilent for wsgi: wsgitemplates
• Example captcha’s, most form libraries have a different way to implement captcha’s
• Uses wsgi middleware to rewrite the form
• The backend application has a checkbox to see if the user is human. Middleware inserts a captcha widget. If theform is submitted the middleware checks the captcha input and on valid input the ’is human’ box is checked. On valid captcha input the form is stored, else an error is shown.
• Probably overkill, more initial effort but can be easy to apply to customer sites.
• Example for how to use wsgi middleware
• See islay.simplecaptcha and islay.hardercaptcha

Multiplayer Plone : Realtime collaboration 

Geir Bækholt

• Jabber/XMPP protocol is used, realtime and asyn
• Presence (status), message, iq (subscription),
• Communication not directly thru Plone, but via javascript
• Scales to large amount of users
• Collaborative writing like Google Docs in ATContent and dexterity
• Probably same diff match patch algorithm as in google docs
• Future: content notifications, async queues with xmpp in plone, conferencing, video and audio chat.
• See jarn.xmpp.buildout 

Killer feature/selling point for intranets and sites with lots of authors.

Clone to Plone

Adam Terrey

• How to move from older ‘systems’ to Plone. Migrating content, it’s pretty hard.
• Funnelweb http://plone.org/products/funnelweb
• A web crawler, uses mr.migrator and transmogrifyer
• Buildout, command line or Plone TTW

• Crawler is configurable; ignore regex, drop content using tal, amount of items, ignore robots meta
• Crawler saves in cache on disk
• Content extraction using xpath
• Restructure content, tidy titles
• Upload to Plone

• What’s not converted: dynamic content, front pages, collections, portlets.
• Diazo is used to create to look and feel of the old site
• Used to migrate big sites, est time to migrate a big site is 3 / 4 days

Making Plone Mobile using Responsive Web Design

Rob Porter

• Responsive design, geared towards lot’s of devices (and screen sizes!)
• Why design websites for mobile? Because there’s a huge increase on mobile device usage.
• See http://bostonglobe.com http://forefathersgroup.com and make the browser window smaller. The layout changes but content stays the same.
• See http://mediaqueri.es
• Define a range of screen resolutions with @media, to cater different devices. Each resolution range has an own layout/style/css.
• See plonetheme.responsivetheme, makes Plone responsive
• Book to buy Responsive Web design – Ethan Marcotte
• First start with smallest screen sizes and then go bigger

Creating websites using responsive desgn better than creating specific iDevice, Android, Blackberry apps.

The future search of Plone

Sally Kleinfelt, Hanno Schlichting and someone from six feet up

• Information retrieval
• Two search engines ZCatalog and Solr
•  ZCTextIndex is very simple, TextIndexNG adds multilingual, better parsing, binary transforms, synonyms.
• Solr, based on Lucene Java search library. RESTful APIs. Powers twitter and wikipedia. An army of engineers working on it
• Solr has way more search features than ZCatalog
• Collective.solr and alm.solrindex are available add-ons. Collective.recipe.solrinstance as buildout recipe
• Solr indexing is not done transactional, it takes a while before the contt is indexed

De PyGrunn ‘Python and friends’ conferentie is gehouden in Het Paleis in Groningen en is bezocht door ruim 80 personen. Voorafgaand aan de conferentie was er een masterclass van Armin Ronacher. De conferentie is gericht op de Python programmeertaal en verwante technologie. Met sprekers uit Nederland en het buitenland was de Python-community goed vertegenwoordigd op PyGrunn. Vanuit Goldmund, Wyldebeast & Wunderliebe heeft Kim Chee Leong een presentatie gegeven over buildout, dit is een tool voor het uitrollen van applicaties.

Goldmund, Wyldebeast & Wunderliebe en Paylogic hebben de conferentie georganiseerd. Vanwege het grote success zal PyGrunn er ook volgend jaar weer zijn.

I use Setuptools 0.6c9 to create Python eggs. It happens that the latest Setuptools won’t play well with subversion 1.6 (shipped with Karmic).

When creating an egg with:

python setup.py bdist_egg

I got this error:

subversion unrecognized .svn/entries format

You can fix this by running a patch from the setuptools team, download the most recent patch from the issue page. Find out where setuptools lives in your site-packages and apply the patch. More details here if you’re not familiar with site-packages and applying patches.

If you want try buildout for Django try this how-to.

Buildout is well known in the Zope/Plone world and can be used in both development and production stages. Because a production mod_python/Django setup is a bit different them I’m used to (using Zope behind Apache), I decided to use buildout to generate a Apache virtual host config.

You can follow these these steps on this wiki for a buildout with mod_python:

https://tracpub.yaco.es/djangobuildout/ (warning! just click thru the SSL errors)