Back to Article Listing

Pylons - First Web

Important

This is not a comprehensive how-to. These are just my notes. They are written from the perspective of someone new to Pylons. Consequently, one may find a few 'rookie mistakes'. While I tried to be pythonic in my examples, not everything may be the most pylonic.

It is assumed that the official Pylons documentation has already been read and basic concepts of MVC are understood.

Some of the packages discussed (snipped from /site-packages/easy-install.pth):

  • Pylons-0.9.5
  • decorator-2.0.1
  • FormEncode-0.7.1
  • Paste-1.3
  • Beaker-0.7.2
  • WebHelpers-0.3
  • Routes-1.6.3
  • docutils-0.4
  • SQLAlchemy-0.3.7
  • AuthKit-0.4.0dev_r88
  • elementtree-1.2.6_20050316
  • Mako-0.1.9dev_r297

Preface

The Project

First -- I want to extend a big thank you to the people behind Pylons. I know that developing this framework is voluntary. They did not have to do it, and certainly did not have to make it available to the world. On top of all the coding, they still take the time to write documentation and answer questions on the mailing lists and IRC. This is a very exciting project and getting better every day. Thank you.

Why this was written

Pylons official documentation is very good, but, some things I learned were through inference or trial & error. While this is a natural progression for learning anything new, these notes may shorten the learning curve for others.

When I was writing my first app, I would continuously look for more articles about creating a Pylons web. I did not care if the topic was already covered elsewhere. Each author may have a way of saying things a little differently, helping me to get the concepts. So, with this, I am just adding noise to the quality Pylons documentation available.

Approach

Starting out with a new web framework (or any new process, for that matter) can be overwhelming. My approach is to start small and simple. Then, with each success, I add layers and complexity. I did not try to put all the neat-o things in the app at once. This "build on my success" approach also helps isolate problems.

I also kept in mind that the first web app will not be the final version (published) of the web app. That first one will be great practice, but may be a little messy. Besides, rebuilding something from scratch ensures you understand how each bit works, reinforces learning, and shorten trouble-shooting time. Note; some of the code examles were simplified for illustration purposes and are probably not the final versions in my app.

Early words to the wise:

  • first, read the official documentation
  • do the tutorials
  • re-read the official documentation
  • read the mailing list archives
  • re-read the official documentation
  • find other examples on the web
  • re-read the official documentation
  • read the source code

Important

Believe it or not... if you are having a problem and can not find the answer in the documentation or in mailing list archives.. read the source code. The answer is there. Of course, it is easier to just ask questions on the mailing list or IRC (and even I find myself sometimes doing that). But know that over time, the more you understand the process, the more you will turn to the source to find answers. A case in point: Authkit. At the time of this writing, the author was in the middle of a code update and some of the documentation, understandably, lagged. No big deal, I had the ultimate docs.. the source code.

With the "build on my success" approach, here is how I slowly worked up the web you are reading now. The main advancements will be sections below.

  • run the 'hello world' web app right out of the box
  • add a basic home page and mark it up
  • add more pages and a menu
  • add external content and 'include' them in templates
  • add verbose debug logging
  • put first draft of web into production
  • convert external content html to reST with pygments
  • add a database and move content in there
  • add an admin page with content edit capability
  • and authentication
  • put second draft of web into production
  • rinse & repeat

Some would call this an iterative approach to software development and release.

'Hello World'

Pylons works right out of the box. Which is great. After I ran through their Getting Started and Quickwiki, I started tweaking some of the original settings.

development.ini

[server:main]
use = egg:Paste#http
host = 192.168.100.49
port = 5050
server_version = MyPasteWSGIServer

Running on localhost (127.0.0.1) is fine for immediate testing, but I knew I wanted to test the web I was building on several platforms (mostly for css). So I wanted each computer on my home network to be able to view it. For that, the web had to run on an external IP. Since I run my own DNS internally, I then created a resolvable address like pylons.mydoamin.com. Now the Linux and Windows computers can just type in that address (and port) to test the web; in addition to the OpenBSD box I am developing on. None of this was necessary, but it was convenient. An added bonus to this exercise was, eventually, this web would be hosted on a box in a datacenter and changing those values early on confirmed all would be well.

You can also set custom Server values in the headers.. if you really wanted to. Additionally, you can use any available, unprivileged, port. Check your /etc/services to find out what is already being used. (e.g. postgresql runs on 5432/tcp by default) Though I do not use shared hosting, I hear that using something other than 5000 is required.

Mako Templates

File system layout

It was time to start thinking about file system layout. Pylons gets you going pretty well, so I wanted to make sure I kept it neat.

project root directory
    |-- README.txt
    |-- data
    |   `-- templates
    |       |-- components
    |       `-- pages
    |-- development.ini
    |-- ez_setup
    |-- myweb
    |   |-- config
    |   |-- controllers
    |   |-- docs
    |   |-- i18n
    |   |-- lib
    |   |-- models
    |   |-- public
    |   |   |-- css
    |   |   `-- images
    |   |-- templates
    |   |   |-- components
    |   |   `-- pages
    |   `-- tests
    |       `-- functional
    |-- production.ini
    |-- setup.cfg
    |-- setup.py
    |-- temp
    `-- test.ini

That is just a snip, so not everything was not included. Some things to note:

  • I am serving static data from Pylons so public has an image directory. I was playing around with several css markups, so I lumped them together too.
  • There are three ini files: development, production, and testing. Some of the settings are different. I will talk about that below.
  • Rather than dump all my Mako files into the root of templates, I break them out into pages and components. Again, details will be below.
  • the data directory is the cache set in the ini file

Having things well structured on the file system helps me.

Warning

Premature optimization is neat to think about but try not to waste cycles doing it until you need it. Serving static data from Pylons is an example. If I ever find myself in the flattering position of being overloaded with web traffic, there are many things I could do to improve response times. I will worry about that when the time comes. Right now, I want a working web.

Inheritance

As Pylons suggest, I changed the template engine to Mako, then went to their site to read up on their docs. One of the first things I applied was inheritance. I had already created a Home controller and matching home.mako template. Realizing that most of the pages for this web will have the same header, footer, etc; I also created a base.mako template. (btw, I use .mako as a file extension. I am not sure it matters)

# -*- coding: utf-8 -*-
<%doc>
    this is the base template that most other pages inherit.
    Common components are brought in here, as well.
</%doc>

<!DOCTYPE html PUBLIC "..">
<html xml:lang="en" xmlns="..">
<head>
<meta bunch of meta junk />

    <title>My Web</title>
    ${ self.stylesheet.show_style(c.style) }
</head>

<body>
<div id="container">
${ self.show_nav() }
${ self.stylesheet.show_header(c.style) }

    <div id="content">
    ${ self.show_content() }

<div id="footnote">
    <p>footer stuff..</p>
</div><!-- close footnote -->

</div><!-- close content -->
</div><!-- close container -->
</body>
</html>


<%namespace name="navigation" file="/components/navigation.mako" inheritable="True"/>
<%namespace name="stylesheet" file="/components/stylesheet.mako" inheritable="True"/>

Here are some points of interest in that file.

def and namespace

Mako's doc style comments help me when I go back to a template and read what I was thinking when I did something.

The page title is static, for now. That could potentially become a dynamic variable. Search Engine Optimization (SEO) is not a concern of mine, right now. Below the title, notice that I have a show_style call that brings in a style sheet and a related header image further down. (these will be removed once I settle on a final stylesheet) These calls, along with show_nav and show_content are components that are directly associated with the namespace declarations at the bottom.

Here is an over view of that process. You tell the template engine that you have written some functions -that return html- and have put those function into a file. You make that declaration as a 'namespace' and give it a name. Then you can use that name to call those functions. They are like Python's include statements. You can even pass variables to the function. Mako makes this so easy and intuitive.. well done, Mako! In the above example, I told the template engine to retrieve the chosen stylesheet from the show_style function that is in the /components/stylesheet.mako file.

Note that show_nav and show_content do not use the namespace directly. They are actually calling functions that will be in the downstream inheritance; the home.mako file. I just declare the manespace here so it will be available to any template that uses base. When you inherit base, you inherit all its namespaces too.. it you declare them as such. You can read more about scope and inheritance in Mako's documentation.

Now the home.mako file.

# -*- coding: utf-8 -*-
<%inherit file="/pages/_base.mako" />

<%def name="show_nav()">${ self.navigation.nav_main('Home') }</%def>

<%def name="show_content()">

        <div id="content_main">
            <h1>Welcome</h1>
                <h2>This is Home page content</h2>
                    <p>blah, blah, blah</p>
        </div><!-- close content_main -->

</%def>

I have simplified it, but, there are some interesting bits here. The main thing contained in the home.mako (in this example) is content within my content_main div. All my templates kinda follow this pattern. I have drawn a line in the sand and it works for me. Having show_content as a defined function, not straight html, seems redundant. But that is what base is expecting and there are many other things going on that are not detailed here. In short, it allows for flexible placement of content on the rendered page. Not all bits of the page, for any given template, may be contiguous.

When 'Home' is chosen from the menu (or typed in the urlbar as /home) routes grabs the request and decides that the home controller should get the call. That controller has been told to call the home.mako file (e.g. via in the index() function). The first thing home.mako does is inherit base.mako, to bring in all the common elements like header and footer; and get its namespaces. The next thing it does is use the navigation namespace declared in base and calls a function to get a personalized navigational menu. It tells that function that it is the Home page calling, so that particular menu item should be highlighted in the html that is returned.

At first blush, this all seems circumlocutive. The namespace is declared in base > Base calls a function in home to get the nav menu > Home calls the actual function (which is in a file in the components directory) and forwards it back to base.. separately from the normal home content. This is done because of the position on the rendered page. The nav menu is not within the content_main section, but is customized for the home page. I originally wanted that nav call in the controller. But it did not mesh with my ideals that generating and rendering html should be in the templates, not with Python code. Besides, the controllers do not call base. They call other pages that inherit base.

That was a quick overview of setting up this template.

Fat Controllers and Skinny Templates

I have read a lot about the separation of church and state (rather: code and html). I am of the opinion that there should not be html in the controllers. But there may be some code (beyond variables) in the templates. My loose rule of thumb is; if code is generating and rendering html, it is OK in the template. But, I try not to get too complex in there.

If a bit of code is going to be used by more than one template, I break that bit out and put it into the components directory for reuse. I make those functions callable by declaring them as defined functions. The navigation menu is an example. This keeps the menu choices in a single place for easy editing. It also gives the flexibility of adding more menu choices in non-public processes e.g. the user has authenticated and can see the 'admin' pages menu choices.

As implied in the section title, I prefer a fat controller and a skinny template. The controller is in charge and the template should just render the html. To that end, I make good use of the c variables mentioned in the Pylons docs. The controller will run code and generate most of the required dynamic data. The controller fills c variables with strings, lists, dicts, db query results, etc; and passes the results to the template. An example would be how you chose this document to view. Routes determines what action you are trying to do. It tells the controller that you have chosen to view a specific article. The controller pulls the document from the database, converts it from native reST to html, and hands that chunk as a c.chosen_article to the template to render. The template does not know, or even care, that the original data was pulled from the file system, a database, or remote repository. That 'dumbness' makes maintaining templates really easy.

Setting a nav menu

Among some the Mako features I applied are the items from the aforementioned navigation.mako

# -*- coding: utf-8 -*-

<%def name="nav_main(page_call)">

<%
remote_user = request.environ.get('REMOTE_USER')

menu_list = [('Home','home'),
            ('About','about'),
            ('Articles','articles'),
            #('News','news'),
            #('Downloads', 'downloads'),
            ]

priv_list = [('Admin', 'admin'),
            ]

if remote_user:
    for extra in priv_list:
        menu_list.append( extra )

%>
    <div id="nav">
        <ul>
        % for link, url in menu_list:
        %     if page_call == link:
                    <li id="current"> <a href="/${url}">${link}</a></li>
        %     else:
                    <li> <a href="/${url}">${link}</a></li>
        %     endif
        % endfor
        </ul>
            <div id="upper_right_corner">
                ## include a logon item in the corner
                % if remote_user:
                     ${ remote_user }
                     <a href="/login/sign_out">Log out</a>
                % else:
                    <a href="/login/sign_in">Log in</a>
                % endif
            </div><!-- upper_right_corner-->
    </div><!-- close nav -->

</%def>

The whole thing is a def, or function definition. This way it is callable by name, with page_call as the variable passed in.

At the top I assign a variable, remote_user, to the current session's remote username. This determines if the user is privileged and what their username is. One could print their name somewhere.. like maybe in the upper_right_corner div.. or something. If remote_user otherwise evaluates false, the code blocks that use it will not execute. I could have listed the a tag only once after the endfor statement but it added an extra line in the source. I do not mind writing it twice here to see purdy source when I look at the raw, rendered html.

The menu_list is a key value pair of (link display name, url name). I have not found a way to connect what is here, what is coded in routes.py, and what is coded in Xcontroller.py. And I am not sure I would want to. When I add a page, I have to edit all three. I have commented out a few items in that menu_list to show that you can plant reminders to yourself.

The first use of my remote_user is to see if I should append any special menu items to the normal menu. I show a simple example, but one could get fancy if your auth checking uses groups and roles.

The next step above is to start building the menu itself. I use an unordered list that the css persuades to show horizontally and to highlight the current page list item choice. After generating and closing that ul, the next bit adds a little div with the appropriate sign-in/out link, depending on the value of remote_user in the current session.

Important

Mako hints that you can get from their docs:

When declaring variables or other operational Python code use <% %>. When just using built in Mako-Python use % without closing the tag. As you can see in my example I put that notation off to the left and do normal indenting for ease of reading. That is not required.

Custom Error Page

Another Mako template I include is a custom error page.

# -*- coding: utf-8 -*-
<%inherit file="/pages/_base.mako" />

<%def name="show_nav()">
    ${ self.navigation.nav_main('Home') }
</%def>

<%def name="show_content()">

        <div id="content_main">
            <h1>Error 404</h1>
                <h2>Specifics</h2>
                    <p>.. add a message to the user..</p>
                    <p>.. and/or add ${ c.variables }..</p>
                    <br /><br /><br /><br />
        </div><!-- close content_main -->

</%def>

This is pretty simple, but it styles the error pages with your own markup. To make it come up instead of the default page, I changed the error controller to point to my template.

return render_response('/pages/error.mako')

Summing up Templates

You get the idea. To add basic pages and navigation to the website:

  • add a controller
  • add a page to render for that controller
  • add that page to the menu list
  • rinse and repeat

These static controller pages are easy. Adding dynamic content within them is a little more challenging.

I already had a couple of articles written and marked up in html. They were published on another site, but I wanted to seed my article library with them. Mako made 'including' them into the articles template page really easy. You can read about it in their docs.

Verbose debug logging

OK, I had a working app serving up a few basic page. But as I add complexity to this app, I will need to know what is going on; beyond what is showing up in my browser.

I am talking about software logging here, not web page access logging. The two are distinct and are declared in different places in the framework. I can not stress enough about the help that logging gives. If you want to know what is going on under the hood, you can have the app spit out some good stuff to a log. While the debug feedback Pylons puts up on the screen is good, I frequently consult log output. There have been times that diagnosing problems may not have been as easy without them.

There is already a "Recipes > Request logging" tips page but that was not up when I did mine. What I needed was simple but consistent logs (below code is formatted for web)

in middleware.py

# YOUR MIDDLEWARE
# Put your own middleware here, so that any problems are caught by the error
# handling middleware underneath

## turn on debugging
import logging
logging.basicConfig(level=logging.DEBUG,
    format='%(asctime)s %(levelname)s MOD: %(module)s NAM: %(name)s \t %(message)s',
    filename='/var/log/my_web/debug.log', filemode='w')

I set logging to DEBUG for development. I also set a custom format that works for me. I like a timestamp, who is writing to the log, then a tab character, and the message. I added the tab because it makes reading logs easier on my eyes.

Not only does Pylons pick up on this logging, I can now use debug statements in my code that will write to the same log, in the same format. One useful reason would be to capture who is trying to log onto the admin page.. and if they were successful or not. I will cover the details in the auth section below.

Having a uniform file format also makes writing scripts against it simpler. I can have a cron job comb over the log and summarize stuff, look for strangeness, etc. That job can then send me email alerts or reports, etc.

I have the log file mode set to w. This way every time this import is called I get a new log. This is fine for development. I will change that to append in production. That way, I will not lose history. To make sure the log does not get carried away, I have included a log rotate routine into my web startup script.

Important

I keep all logs in /var/log.

  • That is where logs go.
  • I specifically partitioned my hard drive with that as a slice all to itself. That way, if I forget to prune logs, or if some process spews out so much stuff that it fills up the partition, it does not bring down my whole box. In that remote scenario, logging may grind to a halt, but everything else will still be running and I will still have a shell to do something about it.
  • I already have existing routines that do stuff (like error reporting, backup, etc) there; and can just fold my pylons logs into them

I keep the pylons web in /var/www.. not in /home/user

  • Again, that is where webs go.
  • There is a similar partitioning scheme
  • Backup routines already comb over that place on the disk, no added work.

I keep all pids in /var/run

  • Again, that is where pids go.
  • On reboots, the OS wipes that directory. Programs starting up do not get confused with cruft laying around.

A simple rule of thumb for me is.. if a knowledgeable sysadmin were to come along after me, where would they expect stuff to be? Logs=>/var/logs, Web=>/var/www, etc. Just because this is not a multi developer project to be implemented within a large enterprise, it does not mean that good practices should be ignored.

Moving to Production

OK.. Keeping with my "build on my success" approach; I had a working, albeit simple, web. It was time to create release 0.1 and move it off my desktop and into the datacenter. This is what Pylons calls 'Deployment'. They give a lot of pointers on the website.

How to serve

On my home network, I ran the web directly from the paste httpserve instance on a non-standard port. But a public web must be available on port 80. This is a privileged port so I can not have paste use it.

Decision constraints:

  • easy
  • effective
  • appropriate for my needs

I do not expect a lot of traffic, so, I have a short list of quick and dirty options to serve my web to the public. If I were expecting heavy traffic I would include many other options.

  1. use my firewall to redirect from port 80 to port 5000

    Advantages: I would not have to involve a stock web server at all, pylons could handle the whole deal. This is quick and easy, and it works. The disadvantage is that ALL web traffic would be pointed to the pylons app. I have other things being served by that box.

  2. use the stock webserver already installed and proxy pass

    Advantages: Not only do I have a web server that comes stock with my operating system. It is already in use.

    One important consideration for me, when dealing with servers in the datacenter: I do not have to install any additional software for httpd. I like to keep my servers lean, clean, and as simple as it allows. All I add is a few lines to the httpd.conf.

    Another consideration: I already have fancy log analyzers running on a cron job each night and producing purdy reports. Even with proxy pass, the webserver still captures all the access logs. My reporting can continue, without changing a line of code.

    Less important, but also a consideration: by keeping it simple, I can move the whole app to another server and know that it will just work. I can leave DNS alone and proxy pass to any box in my colo network.

  3. use the stock webserver already installed and one of the cgi family or mod_python

    The advantage to this is.. for me, none. Why add extra layers, connectors, and complexity when it is not needed? /me ducks.

  4. Install lighttpd or nginx

    Yes, this would be 'hip'. But it would be totally unnecessary under the current conditions. I have all the tools I need in my stock OS. I do not want to add to the headache my sysadmin patch & upgrade cycle.

Of course, no matter which option chosen, I will still need to install python and pylons. So much for lean and clean, right?

To egg or not

The Pylons (paste) documentation spends a lot of time on creating and distributing eggs. Frankly, because that is part of their model. Otherwise, installing this framework and using their software would not be fun and easy. In this specific case, I am a customer of one. If I create an egg and and use easy setup, it is because I wanted to learn how, for when I do distribute to more than one client.

So, I will stick with rsync for now. When this moves to cvs, I will use 'check out'. A side note; once I installed Pylons and all the dependencies on the server, I just rsync'd my local web up to it. I ran 'paster serve production.ini' and it worked. I did not have to run any other set up or install commands. Later, when I added a database, I ran setup, once, to populate it.

Start, Stop, Rotate

My website is not to generate income. It is just for fun. So, if it goes down.. [shrug].. ok.. I will restart it. I am actually a little curious how long it will run; and if it breaks; how come. I may look into monitoring the process automatically down the road.

Since I am lazy (as a good sysadmin tries to be) I wrote two scripts to help with remotely updating the core web. (content is updated within the web). One sits on the server and is executed by the user who runs the pylons web (NOT ROOT!). run_pylons will be familiar to anyone who has written their own rc script.

#!/bin/sh
# script to run the pylons web server
#
# stock httpd is logging hits for now, but here was the command
#    paster serve --daemon --pid-file=$_pid --log-file=$_log $_ini start


_working_dir="/var/www/myweb/directory"
_pid="/var/run/paster.pid"
_log="web-access.log"
_ini="development.ini"

cd $_working_dir

rotate_log() {
        # the deb.log is set in the middleware
        # and this bit rotates the file before start

        _INFILE="/var/log/my_web/debug.log"
        _STAMP=`date "+%Y%m%d_%H%M%S"`
        _OUTFILE=$_INFILE.$_STAMP

        if [ -f $_INFILE ]; then
            cp $_INFILE $_OUTFILE
        else
            touch $_INFILE
        fi
}


case $1 in
  start)
    rotate_log
    paster serve --daemon --pid-file=$_pid $_ini start
    ;;
  stop)
    paster serve --daemon --pid-file=$_pid $_ini stop
    ;;
  restart)
    rotate_log
    paster serve --daemon --pid-file=$_pid $_ini restart
    ;;
  monitor)
    rotate_log
    paster serve --reload  $_ini
    ;;
  *)
    echo $"Usage: $0 { start | stop | restart | monitor }"
    exit 1
    ;;
esac

exit 0

I declare some variables, include a simple debug log rotate function, and interpret what was passed in as a variable. The monitor option is if I am logged onto the box and, for some reason, wanted to watch it running.

Of course, if any script exits without failure it should exit 0.

The above script is available to me from anywhere I can get a shell, with:

ssh user@webserver /path/to/run_pylons restart

This script can also be called from the boot up routines and shut down routines. When patching or upgrading the OS and applications on this box, I have enough to worry about. Gracefully shutting down this app and bringing it back up should happen automatically.

An example, specific to my OS, for boot up and shut down:

# in /etc/rc.local --AFTER the postgresql startup
# 20070701 - pylons web
mkdir -p /var/run/paster
chown webuser:webgroup /var/run/paster/
su -l webuser /path/to/run_pylons start
echo -n ' pylons web started'

# in /etc/rc.shutdown --BEFORE postgresql shutdown
# 20070701 - pylons web
su -l webuser /path/to/run_pylons stop
echo -n ' pylons web stopped'

The other script, update_remote_pylons, is on my desktop and is used to remotely update the server.. until I put this thing into cvs.

#!/bin/sh
# rsync local up to remote server
# if you are doing something that may be gnarly, use stop
# otherwise use restart

_excludes="/path/to/pylons_web_update.excludes"
_local_project="/path/to/local/pylons_web"
_remote_project="webserver:/path/to/web/"
_rsync="/usr/local/bin/rsync"
_rsync_opts="--stats -va --delete --exclude-from "
_user="user@webserver"
_web_com="/path/to/run_pylons"
_ssh="/usr/bin/ssh"

    case $1 in
      stop)
        $_ssh $_user $_web_com stop
        $_rsync $_rsync_opts $_excludes $_local_project $_remote_project
        $_ssh $_user $_web_com start
        ;;
      restart)
        $_rsync $_rsync_opts $_excludes $_local_project $_remote_project
        $_ssh $_user $_web_com restart
        ;;
      *)
        echo $"Usage: $0 { stop | restart }"
        exit 1
        ;;
    esac

exit 0

The rsync excludes mentioned above contains:

*.pyc
localweb/data/
localweb/associations/
localweb/nonces/

Production.ini

For the production web I changed the IP and Port to my specific set up in the datacenter. And, of course, set debug to false. I also use a real email address. When I got my first failure notification I was a little alarmed that something was wrong. Then again, it was pretty cool that I got an email notification right away. For the record, it was my bad coding, not Pylons fault. I fixed my source, re-sync'd, and restarted.

So there we have it. I have a working, albeit, simple pylons web; being served from my datacenter box. It is logging hits through existing channels and debugging to its own log. I can get debug reports sent via email automatically or by running scripts against the debug log. And, I have a way to upload changes and rotate the logs. Not a bad start. I test it for a while and all seems well.

Now to add another layer.

Fun with reST

A while back I had a quest for the ultimate source code format for publishing. After extensive research, I had resolved that no perfect solution existed for translating a base document into every potential output (html, xml, pdf, LaTeX, troff, etc). But SGML and DocBook came pretty close. Many open source projects use this standard.

[shrug] OK.. if it is a standard, I am game. I started using DocBook.. but, cheese & rice, that is a lot of tagging. Who the heck thought this was a good idea!?! I dropped the effort, but did learn a lot about different publishing methods and formats.

In stark contrast, Restructed Text is a brilliant way to mark up documents for publishing. Sure, it has its warts. And I have a few nits about the html it puts out. But it is still years easier than anything else. I looked at markdown, too. It seemed like a pretty good alternative. I exchanged emails with the maintainer of that port (not the original author) but, even he confessed to preferring reST. OK, my decision was confirmed.

Add pygments to this docutils love-fest, and writing documents is almost giddy fun.

Down the road, I knew I wanted to add a database, and the ability to edit content from within the app. To get ready for this, I had to make some preparations. I already had some old articles, that were marked up with html. They were currently being served from the filesystem via Mako includes. They must be upgraded.

  • strip them of all html tags
  • reduce the markup to reST
  • replace the <pre>code snippets</pre> with pygments highlighting

Stripping documents of html is both a pleasure and a pain. I lost a little fine tuned control over layout, but got it done. Now I had to add a layer in the controller to pull the document from the filesystem (just like before) but to push it through docutils before handing it to the Mako template.

I am a little ashamed that my solution is not elegant, but here is my first stab at that hack.

from myproject.lib.base import *

from docutils import nodes
from docutils.parsers.rst import directives
from docutils.core import publish_parts
from pygments import highlight
from pygments.lexers import get_lexer_by_name
from pygments.formatters import HtmlFormatter

normal_fmter = HtmlFormatter()
lineno_fmter = HtmlFormatter(linenos=True)

class ArticlesController(BaseController):

    #for id, category, title, url, show_it in (self.article_list)

    article_list=[(0, 'base', 'base', '_base.txt', True),
                (1,  'Topic1', 'Title1', 'file_source1.txt', True),
                (2,  'Topic1', 'Title2', 'file_source2.txt', True),
                (3,  'Topic2', 'Title3', 'file_source3.txt', True),
                (4,  'Topic2', 'Title4', 'file_source4.txt', True),
                ]

    article_dir='/articles/'
    article_default= article_dir + '_base.txt'

    def index(self, item):
        if ( not item.isdigit() ):
            c.item=0
        if not c.formatting:
            c.formatting='HTML'
        c.article_list=self.article_list
        c.article_chosen=self._get_article(c.item)
        c.article_formatted=self._format_article(c.article_chosen, c.formatting)
        return render_response('/pages/articles.mako')

    def view(self, item, formatting):
        if ( not item.isdigit() ):
            c.item=0
        c.item=item
        c.article_list=self.article_list
        c.article_chosen=self._get_article(c.item)
        c.article_formatted=self._format_article(c.article_chosen, formatting)
        return render_response('/pages/articles.mako')

    def _get_article(self, item):
        article_chosen = self.article_default
        for id, category, title, url, show_it in (self.article_list):
            if int(item)==int(id):
                article_chosen = self.article_dir + url
        return article_chosen

    def pygments_directive(name, arguments, options, content, lineno,
                   content_offset, block_text, state, state_machine):
        try:
            lexer = get_lexer_by_name(arguments[0])
        except ValueError:
            # no lexer found - use the text one instead of an exception
            lexer = get_lexer_by_name('text')
        formatter = ('linenos' in options) and lineno_fmter or normal_fmter
        parsed = highlight(u''.join(content), lexer, formatter)
        return [nodes.raw('', parsed, format='html')]

    pygments_directive.arguments = (1, 0, 1)
    pygments_directive.content = 1
    pygments_directive.options = {'linenos': directives.flag,
                    'cssstyles': 'highlight',
                    'cssclass': 'highlight_me'}
    directives.register_directive('code', pygments_directive)

    def _format_article(self, url, formatting):
        from pylons import request
        environ = request.environ
        file_base = environ['paste.config']['here']
        file_path = file_base + '/myproject/templates/' + url
        f = open(file_path, 'r')
        file_content=f.read()
        if formatting=='HTML':
            content = publish_parts(file_content, writer_name="html")["html_body"]
        elif formatting=='RAW':
            import textwrap
            content='<pre>'
            rawf=open(file_path, 'r')
            for line in rawf:
                w = textwrap.wrap(line, 80)
                if len(w)==0:
                    content = content + ''
                for wrapped in w:
                    content = content + wrapped + ''
            content = content + '</pre>'
        else:
            content = publish_parts(file_content, writer_name="html")["html_body"]
        return content

Like I said, it isn't pretty. But the point of the exercise was to use docutils and pygments to format a document before handing it to the template.

I tested it for a while and it worked.

The next step was to remove the array holding the document meta data, eliminate the file system from the whole process, and store everything in a database.

Add a database

Since I already had a PostgeSQL instance on my home net and one on my datacenter net, having to choose a database was mute. There are other choices, but I will use what I already have, and know.

I had already used a database with Pylons when I did the Quickwiki tutorial, so I kinda had an idea of what was involved. SAContext had iterated a few times, so I thought it was safe to give it a whirl. In a word: nice. As a container for your engines, metadatas, and sessions; it is a solid idea. I am glad that Pylons is considering putting it in.

My document meta data array already had many of the fields I needed to store articles. I added a few more that I found I wanted after using it for a while. I downloaded sacontext.py and put it in my web lib directory. I then made the following changes to my app, embelished from the SAContext documents.

Differences between using SAContext and the default Pylons application:

  • Development.ini contains these lines:
sqlalchemy.default.uri = postgres://user:pword@dbserver.domain.com/db_name
sqlalchemy.default.echo = true
sqlalchemy.default.echo_pool = true
sqlalchemy.default.pool_recycle = 3600
  • models/__init__.py contains the 'sac' object, our tables, and our mapped classes. Instead of pylons.database we use myweb.lib.sacontext, which we've downloaded and put in the app's lib directory.
import sqlalchemy as sa
from myweb.lib.sacontext import PylonsSAContext

sac = PylonsSAContext()
sac.session.clear()

articles = sa.Table("articles", sac.metadata,
    sa.Column("id", sa.Integer, primary_key=True, autoincrement=True),
    sa.Column("category", sa.String(50), nullable=False),
    sa.Column("cat_order", sa.Integer, nullable=False),
    sa.Column("title", sa.String(255), nullable=False),
    sa.Column("pub_date", sa.Date, nullable=False),
    sa.Column("whenchange", sa.DateTime, nullable=False, default='now'),
    sa.Column("show_it", sa.Boolean, nullable=False),
    sa.Column("body", sa.String, nullable=True),
    )

class Article(object):
  def __init__(self, id, category, cat_order, title,
          pub_date, whenchange, show_it, body):
      self.id = id
      self.category = category
      self.cat_order = cat_order
      self.title = title
      self.pub_date = pub_date
      self.whenchange = whenchange
      self.show_it = show_it
      self.body = body

  def __repr__(self):
      return "<Article '%s'>" % self.name

sa.mapper(Article, articles, extension=sac.ext)
  • /websetup.py contains instructions to create the tables and populate it with data.
import paste.deploy

def setup_config(command, filename, section, vars):
    #Place any commands to setup database here.

    conf = paste.deploy.appconfig('config:' + filename)
    conf.update(dict(app_conf=conf.local_conf, global_conf=conf.global_conf))
    paste.deploy.CONFIG.push_process_config(conf)

    # Must call Paste's push_process_config before this!
    from myweb.models import sac, articles, Article

    # Drop any existing table with the same name as ours.
    sac.metadata.drop_all()

    # Create our table.
    sac.metadata.create_all()

    # Insert using ORM.
    Article(None, "Category1", 1, "Title1", '20070701', None, True, "some content")
    Article(None, "Category2", 2, "Title2", '20070701', None, True, "some content")

    # Save all our changes to the database.
    sac.session.flush()

Important

When setting up the database and tables, I use lower case names. That is the way I have always been doing it. If you do set up a PostgreSQL database and the table name is capitalized, then EVERY reference to that table name must be in quotes. SELECT * FROM "Articles". I think quoting table and field names is burdensome. I have read opinions that quoting should always be done; and will admit that I kinda see their point; but I am not going to do it.

So far, SAContext seems pretty reasonable. In the SAContext Flintstones example, he uses SQLite. I am using PostgreSQL, so I had one extra step: I had to actually create the database and a user to access it from the application. Then I could fill it up. Please re-read the PostgreSQL docs if this sounds strange to you.

# from the command line

# create the web app user that will talk to the db
createuser -U postgres-rootuser your-app-username -P
# create the database as that user, for ownership and rights
createdb -U your-app-username your-databasename
# fill it up
paster setup-app development.ini

Make note of the username, password, and database name created and match it to the dburi in the ini file. Once setup-app ran, I did a psql query on the command line. Yep, data is there.

With all the data elements in place I just needed to access it from my articles controller. Here is a brief snip.

def view(self, item, formatting):
    if ( not item.isdigit() ):
        c.item=1
    if not c.formatting:
        c.formatting='HTML'

    sac = model.sac
    sac.session.clear()
    Article = model.Article
    fc = Article.c
    q = sac.query(Article).order_by(['cat_order', 'pub_date desc'])
    c.articles = q
    article_chosen = [x.body for x in q.filter(fc.id==c.item)]
    c.article_formatted = self._format_article(article_chosen[0], c.formatting)

    sac.session.flush()
    return render_response('/pages/articles.mako')

The view action takes the article number and a formatting variable. The latter defaults to html, but self._format_article (declared lower in the controller) will accept 'RAW' to produce raw text output. I guess it could also produce pdf if I told it to. The c variables are:

c.articles - the whole record set, used to create a menu list

c.article_formatted - results from docutils that get handed to template

I tested this for a while and found one nit. The session would sometimes show old data. After I stuck some session.clear() statements around, all seemed well.

I know I need to get more familiar with SAContext and SQLAlchemy. It is just that I have been dealing directly with the database for so many years that the added [ab|di]straction gets in my way a little.

I test for a while and all seems well. This iterative approach seems to be working for me.

Accepting user input

With my documents in the database, I could no longer just bring them up in a text editor to make changes. I wanted to be able to edit them within the app. Even though, in the beginning, I will be the only user; I still wanted to do some validation.

I looked over some of the choices. I read the Formecode docs and liked what I saw. Even though the author humbly writes, "It's not perfect, but it's usable.", I found it to be functional, fast, and fun. Thank you, sir.

I also read up a bit on form generators. I really do not see the point of writing more lines of code in a generator than actual lines for the form. So, until I become a form factory, I will skip that step.

I did, however, like the idea of organizing them as Mako defs. For this exercise I only have one form, so this may seem like a little overkill. But, I wanted get some practice for when I would have many forms, with the potential for reuse. I created form_widgets.mako and put it in my components directory.

# -*- coding: utf-8 -*-

<%def name="form_article(form_action=None, id=None, category=None, cat_order=None,
    title=None, pub_date=None, whenchange=None, show_it=None, body=None)">

<%
    if whenchange:
    whenchange=str(whenchange)[:19]
%>
    ${ h.form(h.url(action=form_action), method='post') }
        <br />id:<br />         ${ h.text_field('id', value=id, disabled=True) }
        <br />category:<br />   ${ h.text_field('category', value=category) }
        <br />cat_order:<br />  ${ h.text_field('cat_order', value=cat_order) }
        <br />title:<br />      ${ h.text_field('title', value=title) }
        <br />pub_date:<br />   ${ h.text_field('pub_date', value=pub_date) }

        <%
            if show_it==True:
                show_it_true=True
                show_it_false=False
            else:
                show_it_true=False
                show_it_false=True
        %>

        <br />show_it:      True ${ h.radio_button('show_it', True, checked=show_it_true) }
                            False ${ h.radio_button('show_it', False, checked=show_it_false) }
        <br />whenchange:<br />   ${ h.text_field('whenchange', value=whenchange, disabled=True) }
        <br />body:<br />    ${ h.text_area('body', content=body, size="100x50")  }
        <br /><br />${ h.submit("Submit to %s"%(form_action)) }
    ${ h.end_form() }
</%def>

I defined the 'widget' as form_article and that is how it will be called from the template that needs it. It can accept values for all the fields, but I default to empty. (I could add real default values there) This allows me to use the form for adds and edits. This is an important distinction. Those are different actions in the controller. So when the form is submitted via http, I need to have the right action called. When I declare the form, it expects a value for form_action. (This may not be the case if I had a better understanding of SAContext and SQLAlchemy.. but it works for now)

I used Pylons form helpers just for practice. Right now it is the same amount of code as regular html, but there is potential in the future. I could actually kinda 'generate' the form. The caption, field name and value all resolve to the same thing. Right now, I just want to get it working. I can get fancy later.

The whenchange field is just a reminder of when the record was last changed. It is a datetime field, but I really do not need any more precision than seconds, so I truncate it for viewing.

The show_it radio button is a toggle for adding it to the article listing menu. I need to evaluate what is being passed and set the form control.

Here is a snipped version of the template that calls the 'widget'.

# -*- coding: utf-8 -*-
<%inherit file="/pages/_base.mako" />
<%namespace name="widgets" file="/components/form_widgets.mako" inheritable="True"/>

<%def name="show_content()">
    <h1>Add, Remove, Edit Articles</h1>
        ${ widgets.form_article(form_action=c.action, id=c.id, category=c.category,
        cat_order=c.cat_order, title=c.title, pub_date=c.pub_date, whenchange=c.whenchange,
        show_it=c.show_it, body=c.body) }
        <br /><br />
</%def>

It creates the namespace and calls the form; filling in any values if the controller filled in any c variables. e.g. like from a database call. It looks too simple to be true, but, believe it or not, this works.

Validation

Now, back to the controller, lets call it article_admin. Here is a snip from that.

@validate(schema=model.FormArticle(), form='index')
def add(self):
    # form submission 'add' is just an action

    if request.environ['REQUEST_METHOD'] == 'POST':
        article_dict = {}
        article_dict['id']          = None
        article_dict['category']    = request.params['category']
        article_dict['cat_order']   = request.params['cat_order']
        article_dict['title']       = request.params['title']
        article_dict['pub_date']    = request.params['pub_date']
        article_dict['whenchange']  = None
        article_dict['show_it']     = request.params['show_it']
        article_dict['body']        = request.params['body']

        # now save the new article it to the database
        sac.session.clear()
        model.Article(**article_dict)
        sac.session.flush()

    # if all is well go to index
    h.redirect_to(action='index', item=None)

@validate(schema=model.FormArticle(), form='index')
def edit(self, item):
    # form submission 'edit' is just an action

    if request.environ['REQUEST_METHOD'] == 'POST':
        # fetch the old record
        sac.session.clear()
        Article = model.Article
        art_old = sac.query(Article).get_by(id=c.item)
        # change the values
        # art_old['id']     = request.params['id']
        art_old.category    = request.params['category']
        art_old.cat_order   = request.params['cat_order']
        art_old.title       = request.params['title']
        art_old.pub_date    = request.params['pub_date']
        #art_whenchange     = request.params['whenchange']
        art_old.show_it     = request.params['show_it']
        art_old.body        = request.params['body']
        # now save the existing article it to the database
        sac.session.flush()

    # if successful, redirect to index
    h.redirect_to(action='index', item=None)

Above are the add and edit actions. They are simple and similar. Retrieve the POST values and save them to the database. The fun part was the @validate decorator hung on top of both of them. Formencode was a lot simpler to use than I first expected. The validator is called before the function and expects two variables; the form to validate, and the rule set to evaluate it against.

Here is the model.FormArticle which is in the models/__init__.py

import formencode

class FormArticle(formencode.Schema):
    allow_extra_fields = True
    filter_extra_fields = True
    id =  formencode.validators.Int(if_missing=True)
    category = formencode.validators.String(not_empty=True)
    cat_order =  formencode.validators.Int(not_empty=True)
    title = formencode.validators.String(not_empty=True)
    pub_date = formencode.validators.String(not_empty=True)
    whenchange = formencode.validators.String(if_missing=True)
    show_it = formencode.validators.String(not_empty=True)
    body = formencode.validators.String(not_empty=True)

That was it. It really is easy to use. The schema listed above has been simplified. It just says that most of the fields are required to be not_empty and that they should be of a certain data type. One could get much more complex with validation and you are encouraged to read the Formencode docs.

So, the validator is called before the function and checks the POST variables against this rule set. If any of them fail then Formencode re-presents the form and inserts notations where action is needed by the user. The add or edit functions are not called if the POST values did not validate. A little css highlighting for the inserted notations and I was finished.

I tested it a while and all seems well.

Add Authentication

A user can edit the content of the articles. Now it is time to narrow that field of users to just the ones I want to have that right. Additionally, I can add fine grained control of which users can do what. I need to Authenticate and Authorize. I decided to give AuthKit a try. Because the author was in the middle of a code update, the documentation was in a bit of a flux. I did my best to read all that was available. I also spent some time in the source code; not only to get syntax specifics, but also to get some of the concepts. Here is my interpretation of AuthKit.

AAA

The triple-A (AAA) protocol is summarized as:

  • Authentication - the user is who they say they are
  • Authorization - what specific rights the user has
  • Accounting - the tracking of the user consumption of resources

For my needs in this example, I really do not have a need for the Authorization. There will be one level of access: allowed/not allowed. When I have specific roles or groups of users, I can easily add layers to the base I establish now.

I did not see that the Accounting bit was really covered, but, I wanted a log of successful and unsuccessful attempts. It will be part of my debug log reporting. This shows the flexibility of AuthKit. You just implement what is needed.

When a user tries to access a restricted area, an alarm is raised. Specifically, this is a webserver 401 or 403 response. The user is prompted to 'sign in'. The results of what is POSTed back from the user is evaluated. If all goes well, some variables are set. These can be reference in your app; so the user does not have to keep signing in.

Who is in charge of what at each stage of the process is completely up to you. AuthKit only jumps in to handle something if and when you tell it to.

You tell AuthKit to listen for restricted alarm codes by setting authkit.setup.enable = true in the development.ini (or whatever ini you use). Just because this is set, it does not mean anything will happen. You have only told AuthKit that those response codes are to be acted on.

In order for AuthKit to start the triple-A process (evaluate who the user is, what rights they have, and make note of any attempts) it needs to have the user 'sign in'. This can be done in a variety of ways. The one you choose is up to you.

  • Basic
  • Digest
  • Form
  • Forward
  • OpenID Passurl
  • Redirect

These are defined well elsewhere. In short, if you decide on Basic or Digest, Authkit can handle authentication. You can define a list of users right in the ini file. The author has some built in classes that does all that, if called. See the docs. This rudimentary method is probably enough for only the simplest of needs.

But.. as mentioned in the docs, most people want to make this process part of the app. They would like the option of fine tuned control and the ability to make the sign in form look like the rest of the app. For this, you need Form; but more probably, Forward.

Forward Method

I chose Forward because of my approach to the whole process in relation to my app. I like the idea of having one place to sign in, as apposed to each controller. I will use an office building as an analogy.

When you come off the street and enter the building, you are free to visit any of the shops on the ground floor, the ATM, the pay phone, the coffee shop, etc. Building Management does not care. But to get in the elevator and go upstairs; you need to sign in at the security desk. They will check your id against a list of people that are allowed upstairs. If security is tight in this building, they may even restrict you to certain floors. If they are doing their job right, they will enter your name and time of day, in a log. If everything checks out, you will get a visitors pass. It may be as simple as a badge you stick on your chest (e.g. REMOTE_USER) or it may be a swipe card that you have to use at each floor you visit. The latter has the added advantage of tracking you movement (an audit or accounting trail) or only allowing certain people to certain areas (e.g. HasAuthKitRole). This also frees the people on each floor from having to re-authenticate you. They trust the person who originally granted you rights and the security desk has enforced them.

The docs imply; when you venture into custom forms, you have to do everything. You are welcome to reference the author's implementation methodology for Basic/Digest. But you do not have to do it that way. As a matter of fact, I am under the naive impression that you can not automatically use those methods without recreating the whole environment in your own code. For my simple purposes, I found that trying to emulate that process to be burdensome. I also found that trying to get at the usernames and passwords from the ini file was harder than it needed to be. I am guessing that once you move into custom forms you drop that notion; and delete that stuff from the ini. Note; I used the terms 'impression' and 'guessing'. That was by design, I am not completely certain. Regardless, I decided to pluck a few of the features from AuthKit to use. I may use more later, but what I have works. Again, this falls squarely in the Pylons philosophy, which is a good thing.

I found that every single example in the docs gloss over the critical point of comparing what the user had input in the auth form to the usernames and passwords in the ini. And because AuthKit is a 'generic' add on, all the examples are from the point of view of a WSGI app.

# Quick and dirty sign in check, do it properly in your code
params['username'] == params['password']

The author figures we are bright enough to figure it out. umm.. thanks for the compliment? Here is how I implemented it. First a simple list of tasks with more detail to follow.

  • add AuthKit specifics to ini file
  • create a 'login' form
  • create a 'login' controller
  • create route entries for those requests
  • create array of allowed users
  • set up rudimentary Accounting
  • on successful sign in, fill an environment variable "REMOTE_USER" and set a cookie.
  • on subsequent page visits, check for "REMOTE_USER" (roles and groups would go here)
  • raise an alarm if access to restricted area is requested

The usernames and passwords will be an array to start with. This is very easy to set up and can be converted to the more conventional database model after I get all the bits working.

How I Auth

There will be two basic ways for a user to arrive at the 'login' page: directly by url; and by trying to access any restricted area. If it is the latter, then after successful sign in, the user will be redirected there.

Some of the specific ini file settings gave me trouble. But between the error codes and browsing the source code, this is what I came up with. In the ini file:

authkit.setup.enable = true
authkit.setup.method = forward, cookie
authkit.setup.catch = 401, 403
authkit.signin = /login/sign_in/
authkit.forward.internalpath = /login/sign_in
authkit.cookie.signoutpath = /login/sign_out
authkit.cookie.secret = some secret
authkit.cookie.name  = cookiename

Here I am informing AuthKit that it is enabled, what method I am using, and what server response codes to catch. I also give direction on sign in and out paths. AuthKit should automatically handle signing out if the user just lands in the sign_out path.

Important

I was having errors when I tried to set request.environ['paste.auth_tkt.set_user'](username). It was brought to my attention that cookie had to be added to the authkit.setup.method in the ini file

Here is a simplified login.mako template.

<%inherit file="/pages/_base.mako" />

<%def name="show_content()">

        <% remote_user = request.environ.get('REMOTE_USER') %>

            <h1>Login Page</h1>
            %if remote_user:
                <h2>You are signed in as ${ remote_user }</h1>
            %else:
                <h2>Message: ${ c.login_message or "(No message)" } </h2>
                <form method="post"> <!--action defaults to same page -->
                    <br />username:
                    <br /><input name="username">
                    <br />password:
                    <br /><input name="password">
                    <br /><br /><input type="submit" value="Log In">
                </form><br /><br />
            %endif

</%def>

This login form follows my other templates by inheriting base and my show_content def. I then try to fill in a local variable remote_user from the environment variable. If the user is already OK, then I tell them and be done with it. If not, I give them any c.login_message that may have been delivered from the login controller and present a basic form.

Here is a simplified login controller.

from myproject.lib.base import *
from logging import basicConfig, DEBUG, INFO, debug, info, warning, error, critical

class LoginController(BaseController):
    def index(self):
        return Response('')

    def sign_in(self):
        # if a submission was made, get it
        if len(request.params) > 1:
            username = request.params.get('username', '').strip().lower().decode("ascii")
            password = request.params.get('password', '').strip()
            # if they are valid, set the cookie and redirect; exit is implied
            # if not.. just continue with the code
            if self._valid(username, password):
                c.login_message="Success, you are logged in"
                request.environ['paste.auth_tkt.set_user'](username)
                redirect_to(request.environ['HTTP_REFERER'])

        # if just arrived or not valid, show form
        c.login_message="You have NOT been authenticated"
        return render_response('/pages/login.mako')

    def sign_out(self):
        c.login_message="You have been logged out"
        if request.environ.has_key('REMOTE_USER'):
            del request.environ['REMOTE_USER']
        redirect_to('/login')

    def _valid(self, username, password):
        # this array can be replaced by db
        myusers = {'soemusername':'someuserpassword'}
        # while checking on credentials, do some simple Accounting
        try:
            if myusers[username] == password:
                debug("LOGON-ALERT: Success username=%s" % username)
                return True
            else:
                debug("LOGON-ALERT: Bad password username=%s password=%s"%(username, password))
                return False
        except:
            debug("LOGON-ALERT: NO username=%s" % username)
            return False

Note that there are no AuthKit imports or includes. This is just a standard controller with two visible methods and one private. Routes will push the user to sign_in, so index() will never get called. However, only the controller can check if you are valid. This is really simple, but contains the necessary bits to get the AuthKit basics working using the Forward method.

The user can now logon. I set the most basic flag "request.environ['paste.auth_tkt.set_user'](username)" used for subsequent auth checks elsewhere in the code. This sets the infamous "REMOTE_USER" environ variable. The only missing piece is raising an alarm if they try to enter a restricted area. Pylons and AuthKit has made this easy.

Important

During the _validate function, I wrote out LOGON-ALERTs to the log file. This is an over-simplified 3rd A in the triple-A protocol: Accounting This will allow me to do reporting from the log.

  • Success: Did the user actually logon or is the username/password compromised?
  • Bad password: Did the user have trouble or is somebody trying to crack it with repeated attempts?
  • No username: Who is doing this and why?

A grep of LOGON-ALERTS from the log will give me a report that I can use or ignore. But, at least I am aware of who and when attempts are made.

Here is a simplified snip from the Admin controller.

from myproject.lib.base import *
from pprint import pformat
from authkit.permissions import NotAuthenticatedError
from authkit.pylons_adaptors import authorize
from authkit.permissions import RemoteUser
from authkit.authenticate import middleware


class AdminController(BaseController):

    @authorize(RemoteUser())
    def index(self):
        return render_response('/pages/admin.mako')

In the restricted area, the Admin page in this case, just hang an @authorize decorator and tell it the method to use to check credentials. I already stated that in this example, I am only using RemoteUser. If you have roles or groups, you would use that method instead.

If a user tries to access the Admin page, an alarm will raise before the page is rendered. If all is well, they will get the admin template. If not, AuthKit already knows to redirect them to the login/sign_in page.

I do not cover moving users, passwords, roles, etc. to a database. That, and user maintenance (add, delete, update) is left to the reader.

OpenID for fun

When I read about OpenID, it seemed like a cool idea. It was especially easy to set up. But I do not think it is meant to be a strong security measure. The user is required to identify themselves and that is it. I see this as a very effective way to eliminate captchas for public submissions; like blog comments. I do not see this useful for Access Control Lists (ACL's). I installed the required packages and gave it a whirl.

# authkit for open_id
authkit.setup.enable = true
authkit.setup.method = openid, cookie
authkit.cookie.name  = cookiename
authkit.cookie.secret = cookiesecret
authkit.openid.store.type = file
authkit.openid.store.config = /tmp/authkit_store
authkit.openid.path.signedin = /login/sign_in
authkit.openid.baseurl = http://127.0.0.1:5000

Here are the settings I used to test it out. With this in place, the @authorize decorator now refers to these settings. I used the built-in AuthKit form and the user is presented with a single input box to type in their DNS entry. If you want to try this out, you will need to read up on the concepts and create an account somewhere.

I would like to know if there is a way to use Forward for ACL and OpenID within the same app. I will have to read up some more.

HTTPS

Now I had a way to edit content from within the web. I also had a way to validate that input. And, a way to limit who could to the editing. The only thing missing was encrypting all those conversations between the browser and the server when in the restricted areas. I believe https is right for my needs, here.

My first thought was to force the protocol from the reverse proxy. But I did not want to have to too many edits in the httpd.conf. I also wanted the app to be kinda self contained. My second thought was to leverage the idea that all restricted area access has to go through the /login page. So, I set up access to those menu items and force the protocol in the url_for.

In the components/navigation.mako

<% myhostname=request.environ['HTTP_X_FORWARDED_HOST'] %>
% if remote_user:
     ${ remote_user }
     ${ h.link_to('Log Out', h.url_for(protocol='http',
        host=myhostname, controller='login',
        action='sign_out', item=None)) };
% else:
    ${  h.link_to('Log In',  h.url_for(protocol='https',
        host=myhostname, controller='login',
        action='sign_in', item=None)) };
% endif

When using url_for, I found that once I included the protocol and host options, it would change the host portion of the url to the IP address. Presumably, this comes from the SERVER_NAME variable, but it still, it was wrong. Rather than hard code the SERVER_NAME (if even possible) in the ini, I will just ask the server, where this process is running, to give me his name. Since I knew that I was going to use a reverse proxy, I had to grab the HTTP_X_FORWARDED_HOST and shove that in.

So now, when a user clicks to sign_in, they are forced to use https. And when they sign_out, it goes back. This process may change if I find something better.

Bootnote

I believe that Python (specifically, Pylons) is a great base for web development.

I apologize for any errors or omissions. If you have any feedback, feel free to drop me a note.



Copyright © 20070703 genoverly
(db datestamp: 20070821)

nautical_flag_icon
Copyright © 2003-2015 genoverly