Laman

New

a

Tampilkan postingan dengan label python. Tampilkan semua postingan
Tampilkan postingan dengan label python. Tampilkan semua postingan

Sabtu

adepends.py, utility for analysing android module dependency

There are thousands of modules within android system. They form a very complicated dependency graph. When we want to learn about a particular module in android system, it's not easy to find out where modules that are dependent on by the module we mainly focus on are located. To make this task easier, I wrote adepends.py, which can be used to analysis dependency relationship between modules. It's capable of:
  1. List modules defined within a directory
  2. Generate a graphviz dot based diagram file to show dependency relationship
To show which modules are defined in a directory, we can use this command: "adepends.py -l DIRECTORY_NAME". For example, if we run "adepends.py -l external/protobuf/" command, we get below output:
libprotobuf-java-2.3.0-micro
libprotobuf-cpp-2.3.0-lite
host-libprotobuf-java-2.3.0-micro
libprotobuf-cpp-2.3.0-full
host-libprotobuf-java-2.3.0-lite
aprotoc
libprotobuf-java-2.3.0-lite

To generate a dependency diagram for a particular module, we can use this command: "adepends.py -o output.dot -m module_name". For example, if we're interested in the dependency diagram for charger module,  we can use this command: "adepends.py -o output.dot -m charger". After the command finished, we have output.dot in current directory. Then we run "dot -Tpng -ooutput.png output.dot" to generate a png file for the diagram. And the diagram is shown below:

Each ellipse represents a module, the top line shows the module name, and the bottom line shows the directory that the module is defined. The arrowed edge represents the dependency relationship between two modules.

Minggu

python decorators

Decorator is a very expressive language feature in python. It helps developers to write cleaner, modular code that is easier to extend and maintain. It also helps implement AOP and decorator pattern in python.

Decorator syntax
To declare a decorator function, we can define a function that takes a another function as argument and returns a function. The usage of decorator is very simple, just add a line begins with '@' symbol and the name of the decorator before the function to be decorated.

 1 def decorator(func):
 2     def new_func():
 3         print "decorator message"
 4         func()
 5     return new_func
 6
 7 @decorator
 8 def foo():
 9     print "hello world"
10
11 foo()

The effect is when we call foo function, besides print "hello world" message, the message "decorator message" is also printed. That is, the decorator function extends foo function's behavior.
The code above is equivalent to:
 

 1 def decorator(func):
 2     def new_func():
 3         print "decorator message"
 4         func()
 5     return new_func
 6
 7 def foo():
 8     print "hello world"
 9
10 foo = decorator(foo)
11 foo()

Decorators can also accept arguments, as long as it returns a decorator function that takes a function as argument.
 1 def decorator_with_arg(arg):
 2     def decorator(func):
 3         def new_func():
 4             print arg
 5             func()
 6         return new_func
 7     return decorator
 8
 9 @decorator_with_arg("arg for decorator")
10 def foo():
11     print "hello world"
12
13 foo()


The example above is equivalent to :

 1 def decorator_with_arg(arg):
 2     def decorator(func):
 3         def new_func():
 4             print arg
 5             func()
 6         return new_func
 7     return decorator
 8
 9 def foo():
10     print "hello world"
11
12 foo = decorator_with_arg("arg for decorator")(foo)
13 foo()
The decorator_with_arg function creates a closure, so that the arg argument can still be used after the decorator_with_arg returned. Since it's possible for a decorator to accept arguments, the decorator's behavior can changed based on the argument passed in. So it's possible to write more flexible code.

A more practical example
Here is a more practical example. We create a tracable decorator which is a debugging utility. It will keep records of the number of times that a function decorated with it is invoked.
 1 trace_log = {}
 2
 3 def tracable(func):
 4     def decorated_func(*arg, **kwarg):
 5         if not trace_log.has_key(func.__name__):
 6             trace_log[func.__name__] = 1
 7         else:
 8             trace_log[func.__name__] += 1
 9         func(*arg, **kwarg)
10
11     return decorated_func
12
13 def print_trace_log():
14     for key, value in trace_log.items():
15         print "%s called %d times"%(key, value)
16
17 @tracable
18 def foo1():
19     print "foo1"
20
21 @tracable
22 def foo2(arg, kwd="keyword arg"):
23     print "foo2"
24     print arg
25     print kwd
26
27 @tracable
28 def foo3():
29     print "foo3"
30
31 foo1()
32 foo1()
33 foo1()
34 foo2(12)
35 foo2(13)
36
37 print_trace_log()

If we run the code, it will prints that foo1 function is called three times and foo2 function is called twice.
As the example showed, the code for implementing tracing is separated from business logic code contained within foo1 and foo2. The maintainability of the program is much higher than if the code for tracing is mixed with business logic code.

Summary
We got several benefits from decorator.
First, it helps achieve a better separation of business logic code and auxiliary code.
Second, it helps finding out where the auxiliary is used because decorator employs a very special syntax.
Third, we can extend or change our business logic without having to change existing code. New code can be implemented as a decorator.


References:
Charming Python: Decorators make magic easy
Decorators for Functions and Methods
PythonDecorators

Rabu

blogporter v1.0.0 released

The first version of blogporter is released. It's a utility for synchronizing posts between different blog service provider.
I wrote this tool because I need to maintain two blogs. It's annoying to have to copy and paste my posts to another blog. And the main reason I have two blogs is that blogger's service is not available in china mainland, due to a well-known reason. But I don't want to give up blogger's tight integration with my gmail account. So, I mainly write blogs here, and then use blog-porter to synchronize the other blog with this one.

Refer to http://code.google.com/p/blog-porter/ for more information about this utility.

The immediate usage of this tool is to sync posts from LiveSpace blog to other blog sites, because LiveSpace blog is soon to be closed. To do this, you can:

  1. download LiveSpace blog data through http://{your_name}.spaces.live.com/Migration/Default.aspx
  2. extract the downloaded zip file
  3. install python, and BeatuifulSoup module
  4. run "python blogporter.py --list-blogs" to find out supported blog provider
  5. run "python blogporter.py --src-type=4 -p{path_to_unzipped_folder} --dst-type={number_stand_for_dst_blog_type} --dst-account={dst_blog_account} --dst-password={dst_blog_password} --startdate=2000-01-01 --enddate=2010-12-31 -v"

This tool still has limitations, mainly includes:
  1. it doesn't sync comments
  2. it doesn't sync category information from LiveSpace
I've tried below ways to work with LiveSpace, but failed to find a perfect solution.

  1. metaweblog api, it's limited to retrieve 20 posts at most. 
  2. rss, it returns even less posts than metaweblog api. May be Google Reader API can helps us get more data through rss, since google cached a lot of historical rss data on their own server. 
  3. livespace backup file, it doesn't contain category information. 

I finally choose livespace backup file to implement LiveSpaceProvider, hope this doesn't bother too much.

debugging python script in ipython

ipython doesn't work with built-in pdb debugger. While I tried to debug a python script with "run -d script.py" within ipython shell, I got bellow error:
AttributeError: Pdb instance has no attribute 'curframe'

To debug script in ipython, we need to use a different debugger, pydb, for example. There are only three steps to setup and use it.
  1. download and install pydb.
  2. start ipython with -pydb argument: ipython -pydb
  3. start debugging with: run -d script.py
It's mandatory to set HOME environment variable to use pydb. On my windows box, there is no HOME environment variable set by default, and I got bellow error:
KeyError: 'HOME'
It can be solved by adding HOME environment variable.

Because pydb use the same set of commands with gdb, it's fairly straightforward to get start with it if you have done some debugging in gdb. This is one reason why I pick it as my debugger.

Reference:
Introducing the pydb Debugger
Debugging in Python

Sabtu

install ipython and readline on mac

ipython is a powerful interactive shell for python. With it, we can tak advantage of python programming language in our daily works.
One of ipython's power is its tab-completion feature that needs readline to work. On mac system, it lacks this library due to license issue. There is a proprietary version of readline in mac, but it doesn't work with ipython.
To install them on mac system, we can simply use easy_install utility, like this:
sudo easy_install ipython readline