Pythonik

Redis patterns | search

2010-11-25T18:41:00.000-08:00

Problem

You want to implement search against user objects stored in redis using Python. Something like querying for all user ids whose username begins with "an".

Solution

Here we have user objects stored in as hashes with "user:obj:" as prefix.

For example

user:obj:3955 {id: 3955, username: 'John', ..}

We need some extra data structures to support our search i.e. (search user objects where username begins with given phrase. So search for jo should match John, Joe and so on. We will use sorted sets of all usernames and will assign every element a score. This score is a float and helps us in finding the matching words.

Some scores for eg.

a -> 0.097
ab -> 0.097098
ac -> 0.097099
bc -> 0.098099

So for above four string if we find strings that has score that is => 0.097 and < 0.098, we find all strings that begins with 'a'

Code

Discussion

This to demonstrate simple redis pattern and using it in Python.

xfce and ubuntuone

2010-08-28T18:34:00.000-07:00

I do like Ubuntu Netbook Remix’s UI. However with 10.04 it’s just gone so unstable for me.

After login when system prompts for keyring secret, UNR environment crashes and drops to gnome. I have to relogin if I need UNR env.
After I removed a few packages it could not just start gnome panel causing a great inconvenience. I guess this is due to Evolution integration with latest Ubuntu. Mail client I like and use is Thunderbird. I cant switch to Evolution.
Initially after 10.04 release it was damn slow to respond, so had to do some work around to get it to acceptable speed.

Considering all that I decided to switch to Xfce. It just worked like charm. But now I also use (and like :) ) UbuntuOne service for my backup. UbuntuOne is not integrated for XFCE. Also you cant do everything from UbuntuOne’s cli.

u1sdtool -q; killall ubuntuone-login; u1sdtool -c # configuration
u1sdtool --create-folder ~/my_data # add folders you want to be synced
u1sdtool --list-folders
u1sdtool --current-transfers

For more details you might want to check Ubuntu One wiki .

Reliance Netconnect Broadband+ on Linux

2010-02-24T18:36:00.000-08:00

It works. Make sure while purchasing you inform them that you use Linux
It’s fast and reliable in Pashan, Pune area
Below config worked for me on Ubuntu 9.10 AND 10.04
There are Linux drivers on the CD but I could not get it working on Ubuntu 9.10.
For activation, I had to use Windows :(

sudo apt-get install usb-modeswitch wvdial

vi /etc/wvdial

[Dialer Defaults]
Phone = #777
Password =
Username =
Baud = 460800
Stupid Mode = 1
New PPPD = 1
Tonline = 0
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Modem Type = Analog Modem
Baud = 460800
Modem = /dev/ttyUSB0
ISDN = 0

All izz well?

2010-01-02T18:39:00.000-08:00

Surely one of the most over-hyped film. What was good? Aamir, Music and the first half laughs. Post interval it becomes a predictable, boring, idiot bollywood movie.

Munnabhai s were certainly better. I would say watch it on TV or atleast don’t pay 3 times higher than usual for tickets like we did. 4 and half stars by critic hmmmm..

Unicode

2009-03-27T12:01:00.000-07:00

Pulling your hairs over some i18n bug or you fix it but are not able to explain what. This is little help in getting fair idea about unicode/codecs/encoding/decoding etc.

Quick tips:

a. It does not make sense to have a string without knowing what encoding it uses.
b. Utf-8 is a way of storing string of Unicode code points.
c. Encoding: Transforming a unicode object into a sequence of bytes
d. Decoding: Recreating the unicode object from the sequence of bytes is known as decoding. There are many different methods for how this transformation can be done (these methods are also called encodings).

Now
Must Read 1. http://www.joelonsoftware.com/articles/Unicode.html
Must Read 2. http://stackoverflow.com/questions/447107/whats-the-difference-between-encode-decode-python-2-x
Continue reading 1: http://farmdev.com/talks/unicode/
Continue reading 2: http://diveintopython.org/xml_processing/unicode.html
Continue reading 3:http://stackoverflow.com/questions/440320/unicode-vs-str-decode-for-a-utf8-encoded-byte-string-python-2-x

Tata Indicom USB Modem on Linux

2009-03-14T03:39:00.000-07:00

cat /etc/wvdial.conf

[Dialer Defaults]
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Modem Type = USB Modem
Baud = 460800
New PPPD = yes
Modem = /dev/ttyACM0
ISDN = 0
Stupid mode = 1
Phone = #777
Password = internet
Username = internet

Don't understand above. Um ok but I am too lazy to explain.

LinkedIn backlash

2009-02-16T23:16:00.000-08:00

Linkedin is one of the few sites that has certainly impressed me with it's clever design. I would rate it very highly for professional networking. It has one very popular feature "recommendations". Well while I am not against recommending or get recommended by, as I have done both in past. But I see people who
think that more and more people they have in their list (no matter how they know well each other professionally) and more recommendations they have received (mostly by requesting others) would make their prospects better. Umm oh, I wonder why are they are madly behind this. I receive a quite
a few requests often. Some morning you check your emails and you see that some colleague in your company 2 years ago sends you a mail with subject "can you endorse me?". And the email says something
similar.

Dear ,
I'm sending this to ask you for a brief recommendation of my work that I can include in my LinkedIn profile. If you have any questions, let me know.

Thanks in advance for helping me out.

-

Now this above guy could be someone I don't know that well how well he/she is skilled. But now I can't deny the request so in a day or two I would look at some other recommendations available for my other LinkedIn friends, copy some matter and send what is requested for. He happily accepts and send me a nice Thank you email. I see people who worked completely unrelated departments and has probably no ability to judge other's work, go praising out of good relationships. May be what should happen on Orkut testimonials or somewhere similar.

Does these people who have tens and hundreds of people in network and so many recommendations have no work other than hopping the jobs and sending such requests.

Next time I interview a guy with many endorsements , I would probably more cautious hiring him.

My open source projects

2009-01-25T01:21:00.000-08:00

Syncer: A event daemon based on Pyro.
Stockie: A personal portfolio manager for an Investor

Will soon write more about these projects.

Qemu networking setup

2008-10-12T12:40:00.000-07:00

          
------------        ----------
|           |      |  Guest  |
| Host  ----+------+-----    |
|      |    | Hub  |     |   |
|      |tap0|      |tap1 |   |
|      |-----+-----+-----|   |
| eth0      |      |         |
|   |       |      |         |
----+-------       ----------
 |
(Internet)

Host
* Add a hub
# vde_switch  -x -d -tap tap0 -tap tap1
* Assign ip to host's nic
# ifconfig tap0 192.168.1.1
* Setup ip forwarding
Modify /etc/sysctl.conf
net.ipv4.ip_forward=1
* Setup masquerading
# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
* Fire qemu
# vdeqemu -m 1024 -localtime /vm//jos_8.04_01/jos_8.04_01.img

Guest
# ifconfig eth0 192.168.1.2
# route add default gw 192.168.1.1
# vi /etc/resolv.com
# ping google.com

Configuring your ubuntu for faster internet access

2008-08-25T04:52:00.000-07:00

While there is a lot already written here my quick howto


$ sudo bash
# apt-get install dnsmasq squid
# echo "listen-address=127.0.0.1" >> /etc/dnsmasq.conf
# echo "no-dhcp-interface=" >> /etc/dnsmasq.conf
# vi /etc/dhcp3/dhclient.conf 
# # ^ uncomment line #prepend domain-name-servers 127.0.0.1;
# vi /etc/resolv.conf  # Add nameserver 127.0.0.1 
# /etc/init.d/dnsmasq restart
# vi /etc/squid/squid.conf 

http_port 3128
visible_hostname localhost

acl all src 0.0.0.0/0.0.0.0

cache_effective_user proxy
cache_effective_group proxy

http_access allow all
icp_access allow all

positive_dns_ttl 1 month
negative_dns_ttl 1 minute
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on

cache_dir ufs /cache 400 16 256
cache_store_log none         


# mkdir /cache # I have this dir on reizerfs partition
# chown proxy.proxy /cache

# /etc/init.d/squid restart

Configure your browser to use 127.0.0.1:8080.
Also read detailed dnsmasq setup article http://ubuntu.wordpress.com/2006/08/02/local-dns-cache-for-faster-browsing/

Youtube flash videos to DivX (on Linux)

2008-08-01T01:43:00.000-07:00

This how I convert flash
I usually use Firefox VideoHelper Addon to download youtube videos. To play them on my Philips DVP5986K DVD player from USB drive, I need to convert it to DivX.
mencoder /home/shon/Desktop/file-864260998.flv -ovc lavc -oac mp3lame -ffourcc DX50 -o out.avi

Using DOT language to produce Flowchart

2008-05-08T12:19:00.000-07:00

much better than struggling with the graphical tools.


shon@ubuntu:~$ cat test.dot

digraph FlowChart {

 node [
         fontname = "Bitstream Vera Sans"
         fontsize = 8
         shape = "record"
     ]

 edge [
         fontname = "Bitstream Vera Sans"
         fontsize = 8
         fontcolor = "Red"
     ]

// all blocks
greet [label="Hello, techie", shape="oval"]
which_os [label="What OS do you use?" shape="diamond"]
like_me [label="Great, me too!", shape="oval"]
which_browser [label="You must be using firefox", shape="diamond"]
ff [label="Cool", shape="oval"]
bye [label="Bye", shape="oval"]

// relations
greet -> which_os
which_os -> like_me [label="I use Linux"]
which_os -> which_browser [label="I use Windows"]
which_browser -> ff [label="Right"]
which_browser -> bye [label="what firefox?"]
}

shon@ubuntu:~$ dot test.dot -Tpng -o test.png && eog test.png

Contract verification in Python

2008-04-16T23:10:00.001-07:00



import zope.interface.verify

class ITest(zope.interface.Interface):
   def foo(arg1): pass
   def bar(): pass   

class Test(object):
   zope.interface.implements(ITest)
   def foo(self): pass

class Test2(object):
   zope.interface.implements(ITest)
   def foo(self, arg1): pass
  
class Test3(object):
   zope.interface.implements(ITest)
   def foo(self, arg1): pass
   def bar(self): pass

for cls in (Test, Test2, Test3):
   try:
       if zope.interface.verify.verifyClass(ITest, cls):
           print "OK: %s correctly implements %s" % (cls.__name__, ITest.__name__)
   except Exception, err:
       print "Error detected with %s's implementation: %s" % (cls.__name__, err)

Getting older, getting better and better!

2008-04-16T23:08:00.000-07:00

Python programming is joy. I was stuck on python 2.3 at my work for long and could not really get chance to explore later versions. Now that I got the opportunity doing re-architecture of the product I started exploring these. I am more than excited looking at deque, groupby, defaultdict and much more ... Also on top of it there exist excellent python softwares like twisted, sqlalchemy, turbogears makes it even more cool.
It's little pity that the language is stll somewhat less recognized than others. Or there are more hyped languages exist.

Anyways Python rocks!

SQLAlchemy elixir: Simple example

2008-04-16T23:07:00.000-07:00

DB Setup

[root@localhost ~]# yum -y install postgresql-python \
postgresql postgresql-server
[root@localhost ~]# /etc/init.d/postgresql start
[root@localhost ~]# /etc/init.d/postgresql status
[root@localhost ~]# su - postgres -c "createuser --createdb \
--adduser shon"
[root@localhost ~]# su - shon # normal user
[shon@localhost ]$ createdb test
[shon@localhost ]$ psql test
test=# \q

Code test_alchemy.py


from elixir import *

metadata.connect("postgres:///test")

class Movie(Entity):
 has_field('title', Unicode(30))
 has_field('year', Integer)
 has_field('description', Unicode)

def __repr__(self): return '' % (self.title, self.year)

metadata.create_all()

def test1():
 m1 = Movie(title="Blade Runner", year=1982)
 m2 = Movie(title="Life is beautiful", year=1980)
 objectstore.flush()
 print m1

def test2():
 print Movie.select()[0]

test1()
# test2()