invisible directshow render window

A common task we may want to achieve while creating a media application is to start media playback on a new thread, as the sample below shows:
http://code.google.com/p/rxwen-blog-stuff/source/browse/trunk/multimedia/directshow/dbg_directshow_thread_issue/dbg_directshow_thread_issue.cpp
In the menu of the sample, there are two items. "Render File" will start playback on main thread of the application. "Render New Thread" will create a new thread to playback. Everything work fine in the first case, but in the second case, the directshow rendering window isn't visible, I can only hear sound.
Then I debugged the application with spy++. The image below shows all windows when the media is rendered on main thread. There is a VideoRenderer window exists on main thread.

In contrast, the image below shows all windows when the media is rendered on a new thread. The VideoRenderer window doesn't exist.

I added two lines of code at the end of Render method to block the new thread so it didn't exit after the media playback started. This time, the render window can be seen. All windows are shown below, in which the VideoRenderer window is there.

Actually, the issue has to do with windows working mechanism. A window's window procedure executes on the thread that creates the window. If the corresponding thread ends, the window will also end. That's why we have to keep the new thread from exiting to keep the VideoRenderer window alive.

[quotation from programming windows by Charles Petzold]
Although Windows programs can have multiple threads of execution, each thread's message queue handles messages for only the windows whose window procedures are executed in that thread. In other words, the message loop and the window procedure do not run concurrently. When a message loop retrieves a message from its message queue and calls DispatchMessage to send the message off to the window procedure, DispatchMessage does not return until the window procedure has returned control back to Windows.
However, the window procedure could call a function that sends the window procedure another message, in which case the window procedure must finish processing the second message before the function call returns, at which time the window procedure proceeds with the original message. For example, when a window procedure calls UpdateWindow, Windows calls the window procedure with a WM_PAINT message. When the window procedure finishes processing the WM_PAINT message, the UpdateWindow call will return controls back to the window procedure.
This means that window procedures must be reentrant. In most cases, this doesn't cause problems, but you should be aware of it. For example, suppose you set a static variable in the window procedure while processing a message and then you call a Windows function. Upon return from that function, can you be assured that the variable is still the same? Not necessarily—not if the particular Windows function you call generated another message and the window procedure changes the variable while processing that second message. This is one of the reasons why certain forms of compiler optimization must be turned off when compiling Windows programs.
In many cases, the window procedure must retain information it obtains in one message and use it while processing another message. This information must be saved in variables defined as static in the window procedure, or saved in global variables.
[/quotation]

When working with directshow, it's not necessary to create a new thread explicitly. It's fine to start playback on main thread because the main thread won't be blocked.

Override CheckMediaType with care

A problem I encountered while developing my source filter's output pin is that inside FillBuffer method, the media type and size of the media sample is not exactly the same as the one I proposed in GetMediaType method. The strange thing is my pin has successfully negotiated with downstream renderer filter (its actual type is Video Mixing Renderer-9 filter), and its input pin has agreed to use the media type my filter proposed. So why the actual media type being used changes?

The answer lies in this document, handling format changes from the video renderer:

Video Mixing Renderer Filter
The Video Mixing Renderer filter (VMR-7 and VMR-9) will connect with any format that is supported by the graphics hardware on the system. The VMR-7 always uses DirectDraw for rendering, and allocates the underlying DirectDraw surfaces when the upstream filter connects. The VMR-9 always uses Direct3D for rendering, and allocates the underlying Direct3D surfaces when the upstream filter connects.
The graphics hardware may require a larger surface stride than the image width. In that case, the VMR requests a new format by calling QueryAccept. It reports the surface stride in the biWidth member of the BITMAPINFOHEADER in the video format. If the upstream filter does not return S_OK from QueryAccept, the VMR rejects the format and tries to connect using the next format advertised by the upstream filter. The VMR attaches the media type with the new format to the first media sample. After the first sample, the format remains constant; the VMR will not switch formats while the graph is running.

During the connecting phase, after both filters' pins have agreed on a media type, they perform allocator negotiation. And the downstream pin proposed a larger biWidth during allocator negotiation. As a result to the new proposal, our pin's QueryAccept method is called. If we don't override it, the default implementation of grandpa class CBasePin comes into play. CBasePin::QueryAccept internally calls CheckMediaType method to see if the newly proposed media type can be accepted. Because our pin's CheckMediaType method's original implementation is too sloppy without examining all fields of the new media type, so we ended in the situation I described at the beginning of the post.

So, the best practice for creating a source filter is we need to handle CheckMediaType carefully. Or we can take the alternative way of overriding the QueryAccept method to handle media type change explicitly.

References:
How to Write a Source Filter for DirectShow
Core Media Technology in Windows XP Empowers You to Create Custom Audio/Video Processing Components

Thoughts on directshow

In this post, I'll summarize some features that make directshow stand out, from a developer's point of view.

The architecture of directshow is:

The core of directshow framework is the box in the center.

1. separation of concerns
Directshow divides a complex media rendering task into a bunch of smaller tasks, organized as different filters. Each filter has its own concentration, e.g., grabbing data from source media file, decoding data, rendering the data on display. It's more natural and easier for our brain to resolve complex problems in smaller steps, one by one. Another benefit of this breakdown structure is we can perform unit test on smaller component more easily.

2. mature abstraction layer
Directshow provides a mature abstraction layer on top of sub-tasks' concrete implementations (Adapter pattern). The existence of the abstraction layer makes our project easier to manage. The cost for dealing with changes can be minimized. For example, we can have one guy tries to implement a hardware based decoder filter. And before his filter is available, other guys in charge of other parts can use a software based decoder filter instead. After the hardware based filter is done, it's hopeful to replace the software based filter without requiring changes on other parts because interfaces of both filters are the same.

3. built-in multimedia functions
Besides being a sophisticated core framework, directshow also contains a lot of commonly used multimedia features like common media type decoders, some splitters, and multi-stream synchronizer. With these features built-in, we developers' life will be a lot easier.

References
MSDN: Directshow System Overview
http://en.wikipedia.org/wiki/DirectShow

directshow debugging tips

1. View graph

When we render a media file, directshow uses intelligent connect to build a working graph for us. It may add some filters implicitly if necessary. From my perspective, it all happens transparently. It's user friendly and powerful. But while debugging, we need to know exactly how is the graph constructed and connected, at runtime. Directshow provided a utility method AddGraphToRot in %WINSDK%\Samples\multimedia\directshow\common\dshowutil.h. We can use it to register our graph to running object table, and then view the graph in graphedit. After the graph has been registered, we select File - Connect to Remote Graph in graphedit to bring up remote filter graph list. Then select the registered graph to view.

2. Keep log

IGraphBuilder exposes a SetLogFile method that can be used to specify a log file. Once set, all of the graph's activities will be saved to the log file, including how filter's are connected, how the graph attempted to connect pins. These information are valuable for debugging filters.

pGraph->SetLogFile((DWORD_PTR)CreateFile(TEXT("C:\\graph_builder.log"), GENERIC_WRITE, 0, NULL, CREATE_NEW, FILE_ATTRIBUTE_NORMAL, NULL));

3. Dump graph

In the log file generated by graph, components (filter and pin) are identified with their address in memory. The friendly name of the component isn't shown which make the log file harder to understand by human. Here is a small utility function that dumps all filters and their pins with corresponding memory address and name, in the format shown below.

http://code.google.com/p/rxwen-blog-stuff/source/browse/trunk/multimedia/directshow/graphdump.cpp

4. open source graphedit replacement
http://blog.monogram.sk/janos/tools/monogram-graphstudio/