At the moment I'm doing a lot with parallelism. In my spare time (ahem) I'm trying to look at foreach and I'm noticing it really needs to be a MapReduce problem. At work I'm trying to migrate some existing processes onto a new framework which basically works a bit like a grid, and distributes across machines (strangely reminiscent to breaking into multiple threads but at a different granularity).  The problem here though is no matter how I multi-thread the processing, adding things to Windows.Forms controls must be done back on the thread which creates the control.

One thing I've been doing is to visualise a dictionary of 29,418 words, and what they convert into when stemmed. The problem I have is very akin to Jeff Atwoods post, Everything is fast for small n. I just wanted to add to a ListView, so lets see

Plan A - listview.Add()

Normally we'd just write

listview1.Add("hello"); 

The problem is when we call that 29,418 times it now takes over 10 mins.

Plan B - AddRange()

Maybe the problem is in adding to the displayed listview, i.e. its trying to refresh its data between each add.

ListView.ListViewItemCollection temp = 
        new ListView.ListViewItemCollection(listview1);
foreach(string hello in strings)
{
    temp.Add(new ListViewItem(hello));
}
listview1.AddRange(temp);

Still takes over 10 minutes, but this time the delay is in

temp.Add(new ListViewItem(hello));

Plan C - AddRange(ListViewItem[])

ListViewItem[] temp = new ListViewItem[](strings.Count);
int index = 0;
foreach(string hello in strings)
{
    temp[index++] = new ListViewItem(hello);
}
listview1.AddRange(temp);

This is a stunning change, it now takes 2 seconds.

The moral of this story

If you are manipulating a ListView then don't use Add() to a ListView or to a ListView.ListViewItemCollection. Instead use the overload of AddRange(ListViewItem[]).

What I find truly interesting is this now becomes a Reduce problem, not a Map issue.