Unicode in Python

It’s a very long time since I have had to deal with binary code translations, but when dealing with HTML and JSON with Python, I need to get to grips with it. The € symbol for instance requires the extended character set support and rss should be written in utf-8. This page collects the notes I made in learning about Python’s support for unicode/ascii translations. Continue reading “Unicode in Python”

Web Site Testing with MaxQ

MAXQ is a web proxy with recording and replay features. It is designed to capture and replay tests. They are quite keen that one understands the terms of use of target sites since many prohibit automated access and these days it would seem defend themselves against it.  This article documents how to run MaxQ and make tests, but not how to successfully run them. (If that’s why you’ve come here, look elsewhere but if you have answers please comment here. ) Continue reading “Web Site Testing with MaxQ”

Converting dates to strings using Python

While working on converting ello.json to rss, I needed to convert the date formats, I needed to do it in Python and found the following code worked, using strptime and strfmtime. I had to augment the date string with the name of the day of the week attribute, so felt it had to via a date data type. (This is an important part of the requirement, since otherwise string manipulation would have been enough.) Finding exampes was hard, so I hope it helps, Continue reading “Converting dates to strings using Python”

Exporting ello.co

ello export a feed as JSON, here’s its name structure http://ello.co/username.json but you need to be logged in, god knows why; if not the file returns a version of the profile data. This post thus deals mainly with how to automate ello’s login, the problem of converting the JSON to XML is dealt with on a post of the same name. This post morphed into a generic web login via scripts.

Continue reading “Exporting ello.co”

Twitter API V1.1 (Deprecated)

This post was written in 2013 and I discovered in May 2018 that most of the links failed. It’s been a while, and it surprises me that my little wiki has lasted longer than the resources I point to, but not here. I could have deleted this post, but chose not to; I have heavily edited it so it’s as much a diary as the technical note it once was.

I missed the announcement that Twitter were upgrading their API. I only discovered it when they ran their “test blackouts”. It has broken. This means my ‘mingle’, friend feed and hence facebook feeds are now without my twitter goodness. The rest of this post lists then contemporary resources to help rescue the feeds. Continue reading “Twitter API V1.1 (Deprecated)”

Planet Planet

I started using Planet in about 2008 2006, it would seem? I rapidly came too “Planet Venus” and this article was written during that time,  and was copied from the snipsnap wiki, itself copied from my sun blog. This article focuses on installation and administration. It includes links to my scripts. Continue reading “Planet Planet”


I returned to Python in Oct 2010.

I have two problems to solve,

  1. converting my bliki to wordpress
  2. scripting for KDB & Hadoop

Notes on the old Bliki

  1. http://davelevy.dyndns.info/snipsnap/space/Dave/Python+datetime, a list of notes
  2. http://davelevy.dyndns.info/snipsnap/space/Dave/Python+2.x, an article about my initial python build on the Cobalt Qube. This is really no longer relevant, and an example of content, that should NOT be brough across.

Notes on File I/O

  1. http://docs.python.org/tutorial/inputoutput.html
  2. http://www.penzilla.net/tutorials/python/fileio/
  3. http://www.techniqal.com/blog/2005/05/17/python-simple-file-read-and-write/

Notes on Date and Time

Python does not load the datetime, calendar or time modules as default. They need to be imported.

  1. http://docs.python.org/library/datetime.html
  2. http://docs.python.org/library/time.html

I also found the following tutorial useful.

There is an example of how to use gmtime and strfdate in my wiki entry, on Snipsnap to WordPress.

Notes on blogs.oracle.com

Originally posted on my sun/oracle blog in Feb 2009, copied here in July 2016.

We also discussed Python. I have been trying to write a game theory solver for a 2×2 formal game. I was representing the game as a dictionary so that I could retrieve game scores using the strategy names. One problem is that two dimensional dictionaries get syntactically cumbersome. I had ended up with a list as the key. In theory it should make the programming easier, where game is a dictionary attached to class instance g.

i.e. score = g.game[(‘decoy’, ‘defend’)]

makes great sense where decoy and defend are blue and red strategies, however, I have usually placed the evaluation of a score in an iteration, and so coding the strategy names is rare e.g.

strategies=['heads', 'tails'];
for s in strategies:
    # some iterated code

It is probably simpler to represent the game as a 2×2 matrix held in a list and to use the classic technique of holding the names of the strategies in an ordered list so we can translate the matrix cell location such as n(1,1) into n(tails,tails) by looking co-ordinates up in one, or two name lists.

score=matrix(strategies.index('heads'), strategies.index('heads'))

This would also have the advantage that I could look for and use the matrix manipulation packages that exist to avoid writing a lot of code. The code would look a lot simpler, and not just because I have put a lot of it in an external package; this is usually a good clue that the answer is correct.

Lesson 1: Be careful when using dictionaries.

Python Path

Actually PYTHONPATH, according to my findings, the python module loader looks for modules in the same directory as the top level file, the PYTHONPATH, the standard library directories and then the contents of any .pth directories.  The last technique is useful for windows and where you want it towards the end of the search order.

Socket Programming

It has to be Python

Don’t ask.

Python and datetime


Here are some links


An example of using datetime is

import time
import datetime

#now.tzinfo = 'GMT'; # fails, now.tzinfo is immutable

print now.ctime() print now.isoformat() print now.tzinfo

but otherwise produces the following output

Fri Jan  2 15:15:07 2009

I performed these tests on a windows XP box. tzinfo is empty is this a windows/configuration thing? I can test this with UNIX. I tried setting the TZ variable before running the program and it makes no difference. I found one reference that says Python datetime doesn’t do TimeZones.

Obviously, using the from statement would change the call syntax

from datetime import datetime

should work