avoid memory leak in osip

I was debugging memory leak bugs recently. The bug was caused by incorrect usage of the osip library. It's not uncommon that we meet problems when we rely on a library or framework that we don't fully understand.


Symptom and debugging
The symptom is our application ran more and more slowly, and eventually crashed. This seemed very likely to be caused by resource leak. And after ran performance monitor against our application, it's further confirmed that memory was leaking. The virtual bytes, private bytes of the process was continuously increasing.
With the help of umdh.exe, we can find out exact lines of code that were leaking memory. It showed all stack traces of currently allocated memory blocks (including blocks that were either being used or leaked, so we must identify which blocks were in use and which were not) at the moment of dumping.


Causes
The causes of the memory leak is mainly caused by not understanding below items well.

• transaction isn't destroyed automatically

Osip doesn't take full responsibility of managing life time of transactions. Though osip invokes callbacks registered with osip_set_kill_transaction_callback when a transaction is to be terminated, the transaction isn't freed automatically. This is supposed to be done by osip users.
The first thought I had is to call osip_transaction_free inside the kill_transaction callback, but it was wrong. Because the transaction is still accessed after the kill_transaction callback returned. So, a possible point to free transactions is to do it at the end of an iteration of the main loop, after all events have been handled.
I just don't get why this important point isn't mentioned in the official document.

• inconsistent resource management model

In osip, there are inconsistency between APIs about how memories are managed. For example, we call osip_message_set_body to set the body of a sip message, this function internally duplicates the string we passed to it. So, we can (and need to) free the string passed to it after the function finishes. But when we want to call osip_message_set_method, be cautious! This function doesn't duplicate the string passed in, instead, it simply references the string we gave it. So, we can't free the string which is now owned by the sip message.
Such inconsistency makes it extremely easy to get confused and write code that either crashes or leaks.

logging with osip

As I posted before, logging is an important debugging means. In order to be truly useful and convenient, the logging module should at lease have two traits:

1. can be turned on and off globally
2. supports the concept of logging level

osip also comes with a mature logging system. Besides the traits I just mentioned, it also enables we  to configure log output destination, which can be a plain file, syslog, or a function pointer to a custom logging function. The function pointer enables us to save the log to any possible storage we prefer, e.g., across network.
There is a tiny bug which prevents us using the function pointer mechanism on windows platform if we compile the osip as dynamic library. The author forgot export osip_trace_initialize_func in osipparser2.def file. So our application will end in unresolved external symbol error if we use this function. To get around this, I added the line at the very end of osipparser2.def:

  osip_trace_initialize_func        @416

To use osip logging, we need to:

1. Compile osip with ENABLE_TRACE macro defined
2. Define ENABLE_TRACE in our application
3. Initialize osip logging module
4. Write log message with:  OSIP_TRACE (osip_trace(__FILE__, __LINE__, OSIP_INFO1, NULL, "log message"));

The wonderful thing is we can easily turn off logging by either undefine ENABLE_TRACE macro, or eliminate the line that initialize osip logging module. We can also trun logging message with specific logging level on and off. Very convenient.

An example is available here:
http://code.google.com/p/rxwen-blog-stuff/source/browse/trunk/protocol/osip_logging/osip_log.cpp

sip elements lifecycle

In sip protocol, the difficult thing is the liecycle of sip elements, including dialog, transaction and message. Though defined as different layers in sip, their lifecycle usually overlaps. Message is the layer at the bottom. Transaction is the layer above message, it's comprised of a sip request message (sent by UAC) and all subsequent response messages (sent by UAS) to the request. A transaction is identified by Branch parameter in Via header. On top of the transaction layer is the dialog layer, which defines a peer-to-peer relationship between participants. Its identifier is the combination of the To tag, From tag and Call-ID.
The lifecycle of elements varies depending on the actual messages exchanged. As an example, Alice called Bob, and Bob accepted the call. During the session initialization process, they exchanged a bunch of sip messages. The lifecycle of transaction and dialog elements are shown below:
(Yellow block and blue block represent the lifecyle of a dialog, indicate Early sate and Confirmed state respectively. Green block represents the lifecycle of a transaction.)

The lifecycle of elements are different if Bob rejected Alice's call. In this case, the ack message belongs to the transaction that created by the invite message, not having its own transaction anymore.

The previous example suffices to show the complexity of sip elements' lifecycle. It's import to understand the lifecycle of elements to implement a mature sip protocol stack.

The essential idea of layered design is to hide lower layer's implementation detail. In sip protocol, though it's designed as layered protocol, the lower layer isn't completely transparent.
Suppose our application runs on top of a sip protocol stack. In an ideal world, our application should only relies on the dialog layer to run. But because dialog's lifecycle doesn't fully cover the lifecycle of transactions in lower layer, our application still has to rely on transaction layer or even message layer. The dependencies on different layers of sip protocol stack make things a lot complicated.
In order to simplify things, many sip protocol stack implementations abstract another higher level layer, "call", which is not defined in sip protocol specification. The existance of this layer successfully decouples our application from those different underlying layers.

getting start with osip

As described in rfc3261, sip is a layered protocol. Its layers are shown in the image below.

Unlike other sophisicated sip libiraies, exosip, pjsip, mjsip, which implement all layers of sip protocol, oSip library doesn't implement transport layer. oSip claims it doesn't provide high level api for controlling SIP session, it only provides api for sip message parsing, sdp message parsing and transaction handling. The api contains implementation for syntax, transaction and transaction unit layer.
So, in order to use oSip in our own application, we need to implement transport layer, and hook the transport layer with the oSip library.

Core structures of oSip
oSip library has four core structures, osip_t, osip_transaction_t, osip_event_t, osip_message_t. osip_t is the context of the sip library runs in. osip_transaction_t defines transaction which composes of a request message and all subsequent response messages related to the request. osip_event_t defines event inside transaction pending on handling. osip_message_t is the mapping from sip message syntax to c structure.

osip_t has four lists of transactions, each corresponds to a kind of transaction. For instance, if our sip application makes a call to another sip application, our application runs as uac (User Agent Client). A transaction is created and added to the osip_ict_transactions ( ict is short for invite client transaction) list. In the application being called, a transaction is also created and added to the osip_ist_transactions (ist is short for invite server transaction).
There are three members, msg_callbacks, kill_callbacks and cb_send_message in osip_t. They store callbacks to be called while handling events of transaction. They are designed as internal members. The proper way to add callbacks is using following apis: osip_set_cb_send_message, osip_set_message_callback and osip_set_kill_transaction_callback.

osip_transaction_t contains many members. Among them, the most important ones are:

• transactionid, which uniquely identifis a transaction
• state, which shows current state of the transaction
• transactionff, a fifo containing all events belonging to this transaction

osip_event_t is used to represent an event pending on processing. It doesn't contain much information. It only has the transactionid of the owning transaction, the type of the event and a pointer to the sip_message_t correlate with the event. An event is usually constructed accompany with a sip_message_t instance. For example, upon receiving a incoming request, we can use osip_parse function to parse the received string to a osip_message_t instance, meanwhile, a osip_event_t instance will be created and returned. osip_event_t instance is only useful after it has been added to a transaction's event fifo. This can be done with osip_transaction_add_event or osip_find_transaction_and_add_event functions.

After events have been added to fifo, we need to call osip_xxx_execute functions to actually handle them. There are four osip_xxx_execute functions, each will handle all events belong to a transaction category. For example, osip_ist_execute function will iterate through all IST transactions in osip_ist_transactions list, loop through all events in each transaction's transactionff, call corresponding callback functions depending on the type of the event.

How to use oSip
To use oSip library, we usually follow below steps:

1. Initialize oSip and setup callback functions
2. Implement transport layer. In most cases, we create a socket based transport layer.
3. Create/add events by parsing incoming messages
4. Call osip_xxx_execute function to process all events

Here is a basic sample:
http://code.google.com/p/rxwen-blog-stuff/source/browse/trunk/protocol/basic_osip_sample/basic_osip_sample.cpp
This sample is a UAS(callee) for windows, and just show the basic steps of using oSip without error handling.

Something useful

• It's strongly recommended not to run time consuming or indefinite ending tasks inside callback functions, otherwise, the osip main procedure will be blocked until the callback finishes.
• oSip has provided a wrapper layer above native system apis to make it portable. In order to use its multi-thread feature (which is almost always necessary), we need to define OSIP_MT macro in our application.
  
• In most cases, we don't need to concern about freeing of transaction & message instances. Their life time are managed by the osip runtime, through state machine.

An basic introduction to SIP

What is SIP
SIP Protocol is one of widely used signaling protocol, with the main purpose of, as its name is abbreviated for, initialization session between several participants. Due to its nature, it's mainly used in VOIP, IM applications.

Basics of SIP
Because SIP is always mentioned together with other multimedia application protocols such as SDP, RTP. They are so mixed up that easily confuse beginners, like me. To make things clear, it's necessary to know SIP has nothing to do with those protocols. It can be used independently with SDP and RTP. SIP is just being lucky to be employed to work with them in many situations.
In short, their relationship is:
SDP describes the capabilities and preferences of a participant. Usually, it's included in the payload of SIP message.
SIP is the message flow that enable two or more participants to negotiate, setup, modify and terminate a session.
RTP is the protocol for transferring audio or video data after the session has been established, it usually relies on the information contains in SDP.

SIP is a text based protocol with request/response transaction model, similar to HTTP. Unlike HTTP, whose request and response always comes in pairs, SIP's request and response doesn't hold a one on one relationship. A SIP transaction may have multiple request and multiple response.
The format of SIP message is similar to HTTP's too. A message is composed of a start-line(includes method, request-uri and sip-version), several headers and a message body.
Like SMTP, the recipient of a message doesn't have to be directly accessible to the sender. A message may hop several times before it reaches the destination with the help of Proxy Server.

To better understand SIP protocol, the document SIP Protocol Structure through an Example provides a thorough illustration and it's good material for getting start.

SIP Libraries
PJSIP is a powerful sip library implemented in c. Besides SIP, it also support SDP, and has strong multi-media processing capability. It exposes APIs at different layers so that users can pick either a convenient or flexible API set at their will. It's also a highly portable library can run many operating systems, including windows and linux. The best thing is it has complete document which is rare in open source project.
MJSIP is a java based sip library with SDP, RTP supporting. It's also a convenient and complete solution for building VOIP application. But I'd say the library still have room to improve the quality, for there are some apparent spelling mistakes.
OSIP is part of of gnu project, implemented in c. By following the "do one thing, and do it well" philosophy, it solely focus on SIP and is a light weight, fast library.

References:
http://www.ietf.org/rfc/rfc3261.txt
http://en.wikipedia.org/wiki/Session_Description_Protocol
http://en.wikipedia.org/wiki/Real-time_Transport_Protocol