Redis patterns | search

Problem

You want to implement search against user objects stored in redis using Python. Something like querying for all user ids whose username begins with "an".

Solution

Here we have user objects stored in as hashes with "user:obj:" as prefix.

For example

user:obj:3955 {id: 3955, username: 'John', ..}

We need some extra data structures to support our search i.e. (search user objects where username begins with given phrase. So search for jo should match John, Joe and so on. We will use sorted sets of all usernames and will assign every element a score. This score is a float and helps us in finding the matching words.

Some scores for eg.

a -> 0.097
ab -> 0.097098
ac -> 0.097099
bc -> 0.098099

So for above four string if we find strings that has score that is => 0.097 and < 0.098, we find all strings that begins with 'a'

Code

Discussion

This to demonstrate simple redis pattern and using it in Python.

See Also

• ZrangebyscoreCommand
  

There are already some good writeups on related topics.

• playnicely
  
• autocomplete
  

Unicode

Pulling your hairs over some i18n bug or you fix it but are not able to explain what. This is little help in getting fair idea about unicode/codecs/encoding/decoding etc.

Quick tips:

a. It does not make sense to have a string without knowing what encoding it uses.
b. Utf-8 is a way of storing string of Unicode code points.
c. Encoding: Transforming a unicode object into a sequence of bytes
d. Decoding: Recreating the unicode object from the sequence of bytes is known as decoding. There are many different methods for how this transformation can be done (these methods are also called encodings).

Now
Must Read 1. http://www.joelonsoftware.com/articles/Unicode.html
Must Read 2. http://stackoverflow.com/questions/447107/whats-the-difference-between-encode-decode-python-2-x
Continue reading 1: http://farmdev.com/talks/unicode/
Continue reading 2: http://diveintopython.org/xml_processing/unicode.html
Continue reading 3:http://stackoverflow.com/questions/440320/unicode-vs-str-decode-for-a-utf8-encoded-byte-string-python-2-x

My open source projects

• Syncer: A event daemon based on Pyro.
• Stockie: A personal portfolio manager for an Investor

Will soon write more about these projects.

Contract verification in Python

import zope.interface.verify

class ITest(zope.interface.Interface):
   def foo(arg1): pass
   def bar(): pass   

class Test(object):
   zope.interface.implements(ITest)
   def foo(self): pass

class Test2(object):
   zope.interface.implements(ITest)
   def foo(self, arg1): pass
  
class Test3(object):
   zope.interface.implements(ITest)
   def foo(self, arg1): pass
   def bar(self): pass

for cls in (Test, Test2, Test3):
   try:
       if zope.interface.verify.verifyClass(ITest, cls):
           print "OK: %s correctly implements %s" % (cls.__name__, ITest.__name__)
   except Exception, err:
       print "Error detected with %s's implementation: %s" % (cls.__name__, err)

Getting older, getting better and better!

Python programming is joy. I was stuck on python 2.3 at my work for long and could not really get chance to explore later versions. Now that I got the opportunity doing re-architecture of the product I started exploring these. I am more than excited looking at deque, groupby, defaultdict and much more ... Also on top of it there exist excellent python softwares like twisted, sqlalchemy, turbogears makes it even more cool.
It's little pity that the language is stll somewhat less recognized than others. Or there are more hyped languages exist.

Anyways Python rocks!

SQLAlchemy elixir: Simple example

DB Setup

[root@localhost ~]# yum -y install postgresql-python \
postgresql postgresql-server
[root@localhost ~]# /etc/init.d/postgresql start
[root@localhost ~]# /etc/init.d/postgresql status
[root@localhost ~]# su - postgres -c "createuser --createdb \
--adduser shon"
[root@localhost ~]# su - shon # normal user
[shon@localhost ]$ createdb test
[shon@localhost ]$ psql test
test=# \q

Code test_alchemy.py

from elixir import *

metadata.connect("postgres:///test")

class Movie(Entity):
 has_field('title', Unicode(30))
 has_field('year', Integer)
 has_field('description', Unicode)

def __repr__(self): return '' % (self.title, self.year)

metadata.create_all()

def test1():
 m1 = Movie(title="Blade Runner", year=1982)
 m2 = Movie(title="Life is beautiful", year=1980)
 objectstore.flush()
 print m1

def test2():
 print Movie.select()[0]

test1()
# test2()