Vanderblog

Source code for IndeterminantProgress

I've cleaned up the source code for the IndeterminantProgress component.   You can download it here: Download progress.zip (4.9K)

 

Here are a few more examples of its usage

Source for the example: Download ProgressExamples.mxml (2.7K)

A flexible indeterminant progress indicator

The Flex ProgressBar isn't terribly exciting, and if you've read my previous post, you'll know that it can cause memory leaks in its "indeterminant" form.  Indeterminant progress indicators are nice for when you don't know how long an activity will take, but would like to give the user some visual assurance that something is happening.

My new component, "IndeterminantProgress", is a highly customizable component with lots of knobs and levers to tweak to give you just the right look for your application.  Here is an example:

I've also created a sample application that allows you to tweak all the parameters and auto generate the matching stylesheets and flex code. IndeterminantProgress sample application

IndeterminantProgress is a relatively simple component that inherits from Canvas. It uses a timer to update its visual state. Unlike ProgressBar, IndeterminantProgress makes sure to remove its timer listener when it is removed from the stage.

If you'd like to use it in your application, please feel free to download my component library. flexcomponents.swc

Tracking down a memory leak – ProgressBar considered harmful

I am currently working on a large Flex application.  It is structured around a document metaphor.  Documents contain pages, and pages contain reports.  Reports are tiled on the page, and generally contain Flex charts.  The user can drag new reports on to the page, move reports around on the page, create/delete pages, and toggle between pages.  It's relatively complex, and there are a lot of moving parts.

I noticed that as I tested the application for longer periods of time, it began to slow down.  I look at Internet Explorer's memory utilization while running the application.  It seemed rather high.  As  Flex applications can be very heavy weight, this was not necessarily a concern.  What was alarming was the fact that as I tabbed back and forth between pages in a document, the memory utilization kept rising.  I hit 500MB, 600MB, 700MB... the sky seemed to be the limit.

I had seen the new profiling tools in Flex Builder 3 at Adobe Max this year, so I downloaded FB3 on my development box and ran the application with the profiling tools. One of the tools is a live memory  watcher.  It will show you the maximum and current instance count of every object in the system, along with their memory footprints.

At first there was nothing obvious, but as I ran the application longer, an object called “ReportPod” bubbled up to the top, with an ever increasing instance count and a massive memory footprint.  ReportPod is a container object that supplies a bit of UI chrome around the previously mentioned charts. At any given time there are at most 10 of these objects being displayed to the user.  As a user tabs between pages. the previous page's ReportPod will be removed from the display list and ReportPods for the new page will be created.  ReportPods are constantly being created, added to the display list, and then eventually remove from the display list.  But it was clear that after they were removed from the display list, they were not being garbage collected as they should be.

Takin' the garbage out

The Flash garbage collector works by counting references.  To simplify a bit, if an object is not referenced by any other object on the display list, it will be garbage collected.  If however, the GC can reach the object via a reference chain, however circuitous, the object will not be garbage collected.  Even a single reference to an object can keep it alive, even after it itself is removed from the display list.  A typical cause of this problem can be event listeners.  Event listeners create “hidden” object references.  For example:

child.addEventListener(MouseEvent.CLICK,clickHandler(event))

This will create a reference from child, to its parent.  This isn't really an issue, as “islands” of objects that only have references to themselves are easily garbage collected.  But something like the following can cause a memory leak:

parent.addEventListener('someEvent;,eventHandler(event))

If this event handler is not remove by calling “removeEventListener”, the parent object will retain a reference to the listening object, in order to be able to call its event handler function.  The listening object will not be garbage collected, even after it is remove as a child object from the parent.  Such situations are relatively rare, and in general, if all you are doing is listening for events on child objects, you don't have to clean up and remove your listeners, but in any other situation, it's good practice to make sure you remove your event listeners before the object is removed from the display list.

I looked for event listener leaks in my ReportPod, and it's child components, and couldn't find anything obvious.  We'd actually gone to quite a bit of trouble to clean up stray listeners, even when we didn't have to.  After several hours of pulling my hair out, I decided to take advantage of a new feature in Flex 2 (and I believe the Flash 9 player).  In Flex 2, you can “reparent” visual components.  You can remove a component from one parent container, and add it to another.

If you can't beat 'em, join 'em

As my ReportPods were not being garbage collected, I decided to create an object pool, a reusable collection of ReportPods, instead of always creating new ReportPods.  When the user tabbed from one page to another, the previous page's ReportPods would be removed from their container, and returned to the pool, and then reused, and reparented as the new page was created.  By reusing them, I controlled the number of instances, and as each page had a maximum of ten reports, there would only ever be ten ReportPod instances total. It wouldn't matter if they were never garbage collected.  The application would us a bit more memory than it should, but memory utilization would not increase without bound.

After dealing with all of the issue involved in initializing and de-intializing pooled ReportPods, I had the application back up and running.  This looked promising at first.  Indeed I now had control over the number of instances of ReportPod.  But it quickly became obvious that ReportPod had not been the cause, but rather a child of the ReportPod, called “PieChart”.  This was a composite component that contained a view stack.  The view stack contained a progress bar and a Flex pie chart component.  While data for the pie chart was loading, the progress bar portion of the view stack would be displayed, when the data was finished loading, the pie chart portion of the view stack would be displayed.

The number of instances of the PieChart was still increasing monotonically, even though it's parent ReportPod was now tamed.  I pulled my hair out looking for event listener leaks in the PieChart, and eventually gave up.  I reasoned that what's good for the goose is good for the gander, and created an object pool for charts.  This was a relatively complex endeavor, and ultimately it failed.  The leak remained.

The exercise in object pooling failed doubly.  Although Flex 2 now supports reparenting, it's simply not reliable.  The repartented components simply didn't work in the same way as freshly created objects.  The most obvious symptom is that view stacks internal to the objects intermittently failed to respond to settings of their “selectedIndex”.  A trace revealed that the selectedIndex was properly set, but on screen the view stack was not displaying the proper stack instance – but this happened only intermittently. There was not predictable pattern, but it was enough to make the pooled objects unusable.

It seemed I would have to figure out what was causing the leak, I just couldn't object pool my way around it.

Keeping it Simple

I went back to basics.  I knew something in the PieChart was causing the leak – something about either its child components or its event listeners.  So I created a very simple pie chart that did away with the view stack and displayed a very simple pie chart with static data.  No memory leak.

I slowly added back functionality to the PieChart component.  I added back the event listeners so that the component would be notified when data was available.  I made the pie chart live so that it displayed real data coming from the back end server.  Still, no memory leak.

Then I added back in the view stack with the progress bar.  Bam!  Memory leak.  I replaced the ProgressBar component with a simple bit of text “Running Report...”.  No memory leak.

It appeared that the ProgressBar component was the cause.  I thought about it a bit, and it makes sense.  As I can't really tell how long the report will run, I set indeterminant=”true” on the progress bar, which results in a progress bar that is constantly animating.  A look at the source code for ProgressBar reveals that it uses a “Timer” object to accomplish this animation.

How do you use a Timer?  You register an event listener for “TimerEvent.TIMER”, and it sends TimerEvents to your event handler periodically, based on the timer interval.  Tracing through the ProgressBar source could I saw that this timer is never stopped, and the event listener for the timer is never removed.  It seemed I'd found an event listener leak.

This is enough to cause the indeterminant ProgressBar to evade memory collection, as the internals of the Flex timer event dispatcher always retains a reference to the progress bar instance, in order to deliver timer events.  But ProgressBar is a DisplayObject, with a “parent”, a reference to its parent container.  And that parent container has a reference to its parent container, etc, etc... Suddenly an entire chain of components becomes reachable by the garbage collector.  Thus the mere presence of a ProgressBar at some level in a visual component hierarchy can cause the entire component to evade garbage collection.  Pretty nasty.  Here's an example – don't leave it running to long, it basically makes my laptop unusable after a few minutes.

http://vanderblog.typepad.com/flex/ProgressLeak.html

This example creates a composite component containing a ProgressBar and an Image control.  The Image control loads a large image from Wikipedia.  A new instance of this component is created every second and added to the application.  The previous instance is removed, and should be garbage collected, but is not, resulting in a rapid increase in memory utilization.

Souce code for Leak Example

Summing up

I will grant that my usage of the ProgressBar was rather unique. Typically they are hosted in a relatively lightweight container, and displayed as a popup.  In this situation the memory leaks caused will be small, and probably never noticed.  I however included the ProgressBar in a view stack with some other very complex components with a large memory footprint, and proceeded to constantly create large numbers of  new instances of these components.  This was a recipe for disaster.

I've learned a lot about diagnosing memory leaks, and that their causes can be extremely obscure.  You can be bitten even if you are doing everything by the book, and meticulously cleaning up your event listeners.  There are no guarantees with the Flex framework – it does contain bugs, and it can contain memory leaks.

The new profiling tools in Flex Builder 3 were a significant help.  The memory profiler makes it much easier to find the proximate cause of memory leaks, but as we saw in my situation, the root cause might not be the most obvious offender, it could be a deeply nested child. There is no magic bullet. 

MAX '07 in review

This year's Adobe MAX was my first.  I picked a good one, it was a record setting MAX, selling out, and beating all previous attendance records.

I have to say, it was a bit overwhelming.  I was interested in about three times as many sessions as I was able to attend.  But I was able to hit most of the Flex/AIR sessions that looked vaguely technically oriented, and picked up a few sessions on design.

Repeat after me, I am a programmer, not a Designer

On the design front I was particularly interested in the Flex/Creative Suite 3 (CS3)  integration points. Adobe seems to think that only designers will be interested in these tools, but we programmers would much prefer to spend a few minutes learning how to edit and export a graphic or Flash asset, than create and maintain hundreds of lines of skinning code and CSS.  I think they underestimate the market for good programmer accessible design tools – primary evidence being the Flex graphical layout tool which is not particularly useful for either designers or programmers.

I was particularly impressed with the Flex plug in/templates for Flash.  Not that I know much about authoring in the Flash tool, but it seems that with the Flex plug in, a little effort in learning the basics of Flash will be well rewarded with the almost effortless ability to reskin my entire app with the the push of a button.

AIR Supply

Adobe is pushing AIR hard.  Ebay desktop, and other examples of AIR apps were trotted out at every opportunity.  Their makers were well represented among the session presenters.  AIR is a very important new tool, and I understand Adobe's desire to promote the hell out of it.  They'd love to take market/mind share from MS and Apple.  Why learn Xcode and Visual C++/.Net when you can use Adobe's very slick widget toolkits, and get rich desktop integration, on both the Mac and PC with the same code base?  Well, Sun lost that very same battle long ago on the desktop, but I think this time around, Adobe might have what it takes.

I am not sure I have much use for AIR at the moment.  It is not a programming panacea, and I think you'd have to think long and hard about why you need an AIR app over a Flex app delivered via a web page.  Yes, AIR apps have some advantages, offline access/local storage, local DB, desktop integration, funky window borders, native windows, and a few others, but they also have some distinct disadvantages.  The #1 thing everybody seems to be forgetting is that they require an installation. 

I thought zero footprint install was a virtue – instant startup, automatic updates, you settings are available everywhere.  AIR apps give this all up.  Well they give up the last one if they store settings locally.  Maybe the new features justify giving this up for particular application domains, but certainly not for all.

Let me give you an example.  I sat in on an “Inspire” session that recounted the experiences of the design team that delivered the award winning Ebay Desktop application.  It's an AIR App, very slick, tightly integrated with Ebay's existing services architecture.  It rocks, it truly deserves the accolades it's received.

But, almost everything this application does relies on calls to the back end to fetch auction items, update auction status, etc...  At one point somebody noticed this and asked if it had an offline mode.  The presenter futzed around with networking a bit, and seemed to have a hard time getting his computer off the network, as WiFi took over when he unplugged the Ethernet cable – he even joked “Man, sometimes it's really hard to get off the network”.  Once he succeeded the application displayed a little “Disconnected” icon, and entirely disabled itself.  There was literally nothing intelligent the app could do without a network connection.

I find it rather ironic that the most lauded AIR application at MAX had almost zero offline functionality, when this is the one of the most highly touted features of the new AIR platform.  “Take your app offline, and synchronize when you have a connection”.  Well, it turns out that this is extremely difficult to do well, even with a slick toolkit, and the time will soon come when being “disconnected” will be an exceedingly rare and foreign state.  “Man, sometimes it's really hard to get off the network” - indeed, in the future it may be darned near impossible.

BPM can be sexy too

One of my clients has been looking at various BPM frameworks, so I thought I should take in a session on Adobe's Business Process Management (BPM) tools.  Unfortunately a technical glitch kept me from getting much out of the session, but I understood the basics.  I was mostly interested in Flex integration, but it appears that it is limited to producing little Flex “applets” that can be used for data entry and or analytics as part of work flows constructed in the Live Cycle ES suite.

The primary focus seems to be around the PDF as a means capturing data from the end user, or sending data to the user as custom generated content.  The user can define a PDF as a data entry form, apply business rules, approval routings, and mappings to an external database or other persistent storage.  The sample we worked through in the class was a PDF leave request form that was routed automagically to the employee's manager.

The design tool is “business oriented” - no, or little code, most things are accomplished by dragging icons onto a canvas and wiring them together with little arrows.  You've probably seen something like this before.

These sorts of products are as old as the hills and not nearly as sexy as all of the other new products Adobe has created in the past few years.  I was designing workflow processes and business rules in PeopleSoft something like 7 or 8 years ago.  But Live Cycle serves an important role in a world where PDFs are increasingly used to capture, store, and transmit data to end users.

Networking

Perhaps I am too much of an introverted geek, but I am always disappointed by the lack of networking opportunities.  Sure, I met a few people at lunch, but they worked in entirely different industries.  I feel this was a bit of a missed opportunity, but I am not sure what I could have done differently.  Oh well.  I am sure this is one of the reasons most companies do not trust their programmers to do their marketing for them.

I did miss the “Birds of a Feather” session, which was probably a much better networking opportunity than chatting up random folks at lunch, but unfortunately it was scheduled a bit too late.  I live in Chicago, and had to get home.

Test 123

Many of the Flex sessions I attended went over ground I was already mostly familiar with – the exception being a couple sessions on Ant/FlexUnit and other automated testing tools.  It's usually hard to get clients to buy into a significant commitment to up front testing and QA, but FlexUnit makes it easy to test Flex apps, at least the non-graphical components of the application.

That's a rather large caveat.  Flex is all about visual experience.  I'd say at least 50% of the bugs I fix over the life cycle of an application are related to visual glitches or inconsistencies.  If you want to test user interaction with visual components, you will need to either write custom code to interface with Adobe's automation API (so that's what automation.swc does), or buy a third party tool.  One of the presenters demonstrated using Mercury's Quick Test Pro to record a user session, and then play it back against a Flex app.  QTP has excellent Flex support, but you are stuck recording scripts to a generic “Windows Application” target for AIR apps.

I also learned about “Antenna”, a nice open source Ant based package of templates for building flex applications.  Antenna understands flex apps, supports cross project dependencies, and makes up for some of the deficiencies in Ant, such as detecting whether or not source files have changed before compiling.  The makers of Antenna have also extended FlexUnit to allow text-based reporting of errors back to a logging server.  This allows FlexUnit test cases to be run in batch mode as part of an automated build/test cycle.

Overall MAX '07 was a very worthwhile experience.  I came away with quite a few new additions to my bag of tricks, saw some cutting edge examples of Flex/AIR apps, and experienced all the latest and greatest Adobe has to offer.