Just got an email from RIM with news and such and noticed the link to the OS center they run on their website to help developers choose which BB OS to target. After looking at the stats, I decided to make a comparison and see how the official stats from RIM differ from what I see within my applications.

Here's the distribution published by RIM (dated March 3-d) I am not sure what they compiled into the March 3-d, i.e. what the timestamp of March 3-d actually means. Was it based on OS tracked for March 3-d, was it for last month, week or such. I couldn't find that information.

And below is the distribution I have compiled from the data for the month of April 2011 (a month worth of data)

The breakdown by OS version is not identical but the picture is pretty much similar, except that my chart is much nicer I've got 6.1 devices (hmmm!) and the 5.0 OS section is a bit larger for my statistics than it is for the official statistics. But again, the official statistics are pretty much 2 month old now and actually since the rest of the sections pretty much coincide with the official statistics, I would assume that the number of OS 5.0 just got larger in these 2 month; which by itself is great news for BB developers. Hooray to that and I will try to keep up with statistics to see how the distribution shifts, maybe another post in couple month or when something interesting happens. Let me know in the comments if you'd like to see something like that in the future and whether it was useful for you.

I do an OTA (over the air) distribution for BlackBerry. Some time ago, I wrote here how to [link id='206' text='setup the mime types for the OTA distribution on Amazon S3']. Luckily now, S3 has got a new and very useful web interface, as well as improved mime type detection. They still get the file types not exactly right, but close enough so that the OTA installation for BlackBerry works on "autodetect" settings just fine.

This is however not about S3, this is about detecting the proper OS version of the BlackBerry OS from the user agent of the browser using a few very simple regular expressions.  The code is tailored to PHP, but you can easily adapt it to any language that supports Perl style regular expressions, or you can adapt the reg. expressions as well based on this simple idea:

       $agent = //get user agent raw string

        if (preg_match("/BlackBerry/i", $agent)) {

            if (preg_match("/6\.0\../i", $agent)) {
                $newRelease['os_folder'] = "6.0";
            }
            if (preg_match("/5\.0\../i", $agent)) {
                $newRelease['os_folder'] = "5.0";
            }
            if (preg_match("/4\.7\../i", $agent)) {
                $newRelease['os_folder'] = "4.7";
            }
            if (preg_match("/4\.6\../i", $agent)) {
                $newRelease['os_folder'] = "4.6";
            }
            if (preg_match("/4\.5\../i", $agent)) {
                $newRelease['os_folder'] = "4.5";
            }
            if (preg_match("/4\.3\../i", $agent)) {
                $newRelease['os_folder'] = "4.3";
            }
            if (preg_match("/4\.2\.1/i", $agent)) {
                $newRelease['os_folder'] = "4.2.1";
            }

            //Form your download URL here

        } else {
            //This is not BlackBerry user agent
        }

Nothing complicated, but this should give a start point to any one tying to serve appropriate bundle via OTA.

If you were to use S3, then it would basically be:

// The main link to S3
$downloadURL = "https://s3.amazonaws.com/YOUR_BUCKET/releases/" . $newRelease['version_folder'] . "/" . $newRelease['os_folder'] . "/YourAppName.jad";

Post your thoughts/corrections in the comments!

In some Java implementations/SDKs the Color class does not have the

int Color.argb(int, int, int, int)

method.

One SDK that, as far as I know, does not have this method is the [link id='75' text='BlackBerry SDK'].

This code snippet can be used to perform just that:

public final class ColorBB {


	public static int argb(int A, int R, int G, int B){		
		byte[] colorByteArr = { (byte) A, (byte) R, (byte) G, (byte) B };
		return byteArrToInt(colorByteArr);
	}

	
	public static final int byteArrToInt(byte[] colorByteArr) {
		return (colorByteArr[0] << 24) + ((colorByteArr[1] & 0xFF) << 16) + ((colorByteArr[2] & 0xFF) << 8) + (colorByteArr[3] & 0xFF);
	}

}

Let me know in the comments if you have any improvement/modification ideas.

Great collection of design rules by Eric S. Raymond
Eric S. Raymond

• Rule of Modularity: Write simple parts connected by clean interfaces.
• Rule of Clarity: Clarity is better than cleverness.
• Rule of Composition: Design programs to be connected to other programs.
• Rule of Separation: Separate policy from mechanism; separate interfaces from engines.
• Rule of Simplicity: Design for simplicity; add complexity only where you must.
• Rule of Parsimony: Write a big program only when it is clear by demonstration that nothing else will do.
• Rule of Transparency: Design for visibility to make inspection and debugging easier.
• Rule of Robustness: Robustness is the child of transparency and simplicity.
• Rule of Representation: Fold knowledge into data so program logic can be stupid and robust.
• Rule of Least Surprise: In interface design, always do the least surprising thing.
• Rule of Silence: When a program has nothing surprising to say, it should say nothing.
• Rule of Repair: When you must fail, fail noisily and as soon as possible.
• Rule of Economy: Programmer time is expensive; conserve it in preference to machine time.
• Rule of Generation: Avoid hand-hacking; write programs to write programs when you can.
• Rule of Optimization: Prototype before polishing. Get it working before you optimize it.
• Rule of Diversity: Distrust all claims for "one true way".
• Rule of Extensibility: Design for the future, because it will be here sooner than you think.

I have created the very simple [link id='124' text='text formatter plugin for DokuWiki'] mainly to solve the problem of pasting some text with line breaks. DokuWiki by default ignores these line breaks and this was a problem for me. Without knowing any better I quickly put together the [link id='124' text='wpre plugin'] which simply wraps whatever you want with <pre> tag. I later learned that there is a better plugin for this. It's called linebreak. It works fine for me, even with the latest DokuWiki release, but it seems that the author suggest to use xbr plugin instead.

Using the latest version of Google Goggles on Android, you can automatically solve Sudoku. This is what a little creativity paired with technology does - it brings fun and innovation!

Check out the video of this in action!

The rumors have it that Android 2.4 codenamed Ice Cream is expected this summer in June-July and the release will be announced at the Google/IO.

What's next in store for Android releases? Will the next release going to be named "Jello" or maybe "Jam"? I personally vote for "Jello"

I wrote yesterday about [link id='434' text="my (mostly) negative experience working with these companies"], but some of you might give it a try and maybe your results will be much better or maybe things have actually improved lately. In any case, the code here should get you up to speed covering most of the dirty work of working with these companies' APIs.

I will post 4 classes here:

• AdGrabber -  main abstract class to give all other specific grabbers structure and save some repeating code
• MillenialAdGrabber - MillennialMedia specific grabber
• NexAgeAdGrabber - right, NexAge specific grabber
• SomaAdGrabber - the SMAATO ad grabber. Their ad platform is dubbed SOMA, hence the name

If you are using this code, please give an attribution when applicable with a link to this post.

AdGrabber

/**
 * Abstract class for ad unit grabbing. Inheriting subclasses will implement ad serving platform specific web service interfaces.
 *
 * Copyright: Creative Common Attribution http://creativecommons.org/licenses/by/3.0/
 */
public abstract class AdGrabber {

	protected HashTableExt _adFieldPropertyBag;

	public AdGrabber(HashTableExt adFieldPropertyBag) {
		if (adFieldPropertyBag == null) {
			_adFieldPropertyBag = new HashTableExt();
		} else {
			_adFieldPropertyBag = adFieldPropertyBag;
		}
	}

	/**
	 * Inheriting classes have to implement the grabbing and the parsing of the ad from the appropriate ad serving web service
	 *
	 * @return
	 * 			- an {@link Ad} object populated with the values parsed from the appropriate ad serving web service
	 */
	public Ad grabTheAd() {
		try {
			Document adResponseDoc = StreamUtils.parseRemoteXML(getRequestURL(_adFieldPropertyBag));
			return parseServerResponse(adResponseDoc);
		} catch (Exception e) {
			//Log error
		}
		return new EmptyAd();
	}

	protected abstract String getRequestURL(HashTableExt adFieldPropertyBag);

	protected Ad parseServerResponse(Document doc) {
		Ad ad = null;

		if (doc == null) {
			return ad; // basically = return null
		}

		try {
			if (gotProperAd(doc)) {
				if (shouldParseAsImageAd(doc)) {
					ad = parseImageAd(doc);
				} else {
					ad = parseTextAd(doc);
				}

				ad.setNetwork(StringUtils.getClassName(this));

			} else { // did NOT get a proper ad
				ad = new EmptyAd();
			}

		} catch (Throwable t) {
			//Log error
		}

		return ad;
	}

	protected abstract boolean gotProperAd(Document doc) throws Exception;

	protected abstract TextAd parseTextAd(Document doc) throws Exception;

	protected abstract ImageAd parseImageAd(Document doc) throws Exception;

	protected abstract boolean shouldParseAsImageAd(Document doc) throws Exception;

}

MillenialAdGrabber

/**
 * Ad grabber that interfaces with Quattro Wireless service
 *
 * Copyright: Creative Common Attribution http://creativecommons.org/licenses/by/3.0/
 */
public class MillenialAdGrabber extends AdGrabber {

	/* ------------------------------------------			PUBLISHING CONSTANTS			------------------------------------------*/

	private static final String APP_ID = "YOUR_APP_ID";

	/* ------------------------------------------											------------------------------------------*/

	public MillenialAdGrabber(HashTableExt adFieldPropertyBag) {
		super(adFieldPropertyBag);
	}

	/**
	 *
	 * @param adFieldPropertyBag
	 * @return
	 * 			- The URL to be sent to the advertising web service
	 */
	protected String getRequestURL(HashTableExt adFieldPropertyBag) {
		StringBuffer sb = new StringBuffer("http://ads.mp.mydas.mobi/getAd.php5?");

		sb.append("apid=" + APP_ID);

		sb.append("&auid=" + BBInfo.getUniqueDeviceHash());

		sb.append("&uip=" + URLTools.urlEncodeBBForum(BBInfo.getIP())); // device IP

		sb.append("&ua=" + URLTools.urlEncodeBBForum(BBInfo.getUserAgent())); // USER AGENT

		return sb.toString();
	}

	/* ------------------------------------------------				AD response PARSING			------------------------------------------------ */

	protected boolean gotProperAd(Document doc) throws Exception {
		NodeList nl = ((NodeList) doc.getElementsByTagName("ad"));
		if (nl != null) {
			return nl.getLength() > 0;
		} else {
			return false;
		}
	}

	protected boolean shouldParseAsImageAd(Document doc) throws Exception {
		if (BBInfo.isTallScreen() || _adFieldPropertyBag.get(AdAllocManager.FETCH_IMAGE_ADS) != null) {
			if (!StringUtils.isEmpty(XML.getTextFromNode(doc.getElementsByTagName("url").item(0)))) {
				return true;
			}
		}

		return false;
	}

	protected TextAd parseTextAd(Document doc) throws Exception {
		TextAd ad = new TextAd();
		parseCommonAdValues(ad, doc);

		//parse the TEXT of the banner
		ad.setText(XML.getTextFromNode(doc.getElementsByTagName("altText").item(0)));

		return ad;
	}

	protected ImageAd parseImageAd(Document doc) throws Exception {
		ImageAd ad = new ImageAd();
		parseCommonAdValues(ad, doc);

		//parse the IMAGE of the banner
		String imageURL = XML.getNodeTextCumulative(doc.getElementsByTagName("url"));
		EncodedImage bannerImage = EncodedImage.createEncodedImage(StreamUtils.readRemoteStream(imageURL).getBytes(), 0, -1);
		ad.setImage(bannerImage);

		return ad;
	}

	/**
	 * Parse common values from the responce like the:
	 *  - tracking pixels/beacons
	 *  - the click URL
	 *
	 * @param ad
	 * @param doc
	 */
	private void parseCommonAdValues(Ad ad, Document doc) throws Exception {
		// CLICK URL
		ad.setClickURL(XML.getNodeTextCumulative(doc.getElementsByTagName("clickUrl")));
	}

}

NexAgeAdGrabber

/**
 * Copyright: Creative Common Attribution http://creativecommons.org/licenses/by/3.0/
 */
public class NexAgeAdGrabber extends AdGrabber {

	public NexAgeAdGrabber(HashTableExt adFieldPropertyBag) {
		super(adFieldPropertyBag);
	}

	/* ------------------------------------------			PUBLISHING CONSTANTS			------------------------------------------*/

	private static final String SITE_ID = "YOUR_SITE_ID";

	/* ------------------------------------------											------------------------------------------*/

	/**
	 *
	 * @param adFieldPropertyBag
	 * @return
	 * 			- The URL to be sent to the advertising web service
	 */
	protected String getRequestURL(HashTableExt adFieldPropertyBag) {
		StringBuffer sb = new StringBuffer("http://admax.nexage.com/adServe?");

		sb.append("dcn=" + SITE_ID);

		sb.append("&pos=" + getAdPosition(adFieldPropertyBag));

		sb.append("&ip=" + URLTools.urlEncodeBBForum(BBInfo.getIP())); // device IP

		sb.append("&ua=" + URLTools.urlEncodeBBForum(BBInfo.getUserAgent())); // device USER AGENT

		sb.append("&f=" + "xml");

		sb.append("&u(id)=" + BBInfo.getUniqueDeviceHash());

		return sb.toString();
	}

	private static String getAdPosition(HashTableExt adFieldPropertyBag) {
		if (adFieldPropertyBag != null && adFieldPropertyBag.get(AdAllocManager.AD_POSITION) != null) { //if we specified the ad position - return the specified ad position
			return (String) adFieldPropertyBag.get(AdAllocManager.AD_POSITION);
		} else {
			// otherwise, use the "imageonly" or "textonly", depending on the display size, or manual override
			return (BBInfo.isTallScreen() || (adFieldPropertyBag != null && adFieldPropertyBag.get(AdAllocManager.FETCH_IMAGE_ADS) != null)) ? "imageonly" : "textonly";
		}
	}

	protected boolean gotProperAd(Document doc) throws Exception {
		Node adsNode = doc.getElementsByTagName("ads").item(0);
		if (adsNode != null) {
			int adCount = Integer.parseInt(XML.getAttributeValue(adsNode, "count"));
			return adCount > 0;
		} else {
			return false;
		}
	}

	protected ImageAd parseImageAd(Document doc) throws Exception {
		ImageAd ad = new ImageAd();
		parseCommonAdValues(ad, doc);

		//parse the IMAGE of the banner
		String imageURL = XML.getAttributeValue(doc.getElementsByTagName("ad:content").item(0), "url");

		EncodedImage bannerImage = EncodedImage.createEncodedImage(StreamUtils.readRemoteStream(imageURL).getBytes(), 0, -1);
		ad.setImage(bannerImage);

		return ad;
	}

	protected TextAd parseTextAd(Document doc) throws Exception {
		TextAd ad = new TextAd();
		parseCommonAdValues(ad, doc);

		//parse the TEXT of the banner
		ad.setText(XML.getTextFromNode(doc.getElementsByTagName("ad:text").item(0)));

		return ad;
	}

	protected boolean shouldParseAsImageAd(Document doc) throws Exception {
		Node node = doc.getElementsByTagName("ad:group").item(0);
		String adType = XML.getAttributeValue(node, "type");

		return adType.toLowerCase().compareTo("banner") == 0;
	}

	/**
	 * Parse common values from the responce like the:
	 *  - tracking pixels/beacons
	 *  - the click URL
	 *
	 * @param ad
	 * @param doc
	 */
	private void parseCommonAdValues(Ad ad, Document doc) {
		// CLICK URL
		ad.setClickURL(XML.getTextFromNode(doc.getElementsByTagName("link").item(0)));

		// TRACKING PIXEL URLs - get only ones from the FIRST  node (in case there are more than 1 ad in the response, only the first should be processed and the rest ignored)
		Node adEventsNode = doc.getElementsByTagName("ad:events").item(0);
		NodeList nl = adEventsNode.getChildNodes();

		for (int i = 0; i < nl.getLength(); i++) {
			// If the tracking is of the "display" type - it is a tracking PIXEL URL
			if (XML.getAttributeValue(nl.item(i), "type").toLowerCase().compareTo("display") == 0) {
				String nodeName = nl.item(i).getFirstChild().getNodeName();
				if (nodeName.toLowerCase().compareTo("link") == 0) {
					String trackingPx = XML.getTextFromNode(nl.item(i).getFirstChild());
					ad.addTrackingPixelURL(trackingPx);
				}
			}
			// otherwise, if it is of the "click" type - it is a CLICK tracking URL
			else if (XML.getAttributeValue(nl.item(i), "type").toLowerCase().compareTo("click") == 0) {
				String nodeName = nl.item(i).getFirstChild().getNodeName();
				if (nodeName.toLowerCase().compareTo("link") == 0) {
					String trackingPx = XML.getTextFromNode(nl.item(i).getFirstChild());
					ad.addClickTrackingURL(trackingPx);
				}
			}
		}
	}

}

SomaAdGrabber

/**
 * Copyright © Comitic Software
 */
public class SOMAAdGrabber extends AdGrabber {

	public SOMAAdGrabber(HashTableExt adFieldPropertyBag) {
		super(adFieldPropertyBag);
	}

	/* ------------------------------------------			PUBLISHING CONSTANTS			------------------------------------------*/
	private static final String AD_SPACE = "YOUR_AD_SPACE";

	private static final String PUBLISHER_ID = "YOUR_PUB_ID";

	private static final String SOMA_API_VER = "somaapi-318"; //This is the version of the API I used

	/* ------------------------------------------											------------------------------------------*/

	/**
	 *
	 * @param overrideImageAds
	 * @return
	 * 			- The URL to be sent to the advertising web service
	 */
	protected String getRequestURL(HashTableExt adFieldPropertyBag) {
		StringBuffer sb = new StringBuffer("http://soma.smaato.net/oapi/reqAd.jsp?");

		sb.append("adspace=" + AD_SPACE);

		sb.append("&pub=" + PUBLISHER_ID);

		sb.append("&client=" + SOMA_API_VER);

		sb.append("&devip=" + URLTools.urlEncodeBBForum(BBInfo.getIP())); // device IP

		sb.append("&device=" + URLTools.urlEncodeBBForum(BBInfo.getUserAgent())); // device USER AGENT

		String adMode = (BBInfo.isTallScreen() || (adFieldPropertyBag!=null && adFieldPropertyBag.get(AdAllocManager.FETCH_IMAGE_ADS) != null)) ? "all" : "txt"; // "all" = IMAGE and TEXT; "txt" = TEXT only
		sb.append("&format=" + adMode);

		sb.append("&ownid=" + BBInfo.getUniqueDeviceHash());

		sb.append("&responce=" + "xml");

		return sb.toString();
	}

	protected boolean gotProperAd(Document doc) throws Exception {
		//return ((NodeList) doc.getElementsByTagName("ads")).getLength() > 0 && canProcessTheAd(doc);
		NodeList nl = ((NodeList) doc.getElementsByTagName("ads"));
		if (nl != null) {
			return nl.getLength() > 0  && canProcessTheAd(doc);
		} else {
			return false;
		}
	}

	private boolean canProcessTheAd(Document doc) {
		Node node = doc.getElementsByTagName("action").item(0);
		String actionType = XML.getAttributeValue(node, "type");
		return actionType.toLowerCase().compareTo("link") == 0;
	}

	protected ImageAd parseImageAd(Document doc) throws Exception {
		ImageAd ad = new ImageAd();
		parseCommonAdValues(ad, doc);

		//parse the IMAGE of the banner
		String imageURL = XML.getNodeTextCumulative(doc.getElementsByTagName("link"));

		EncodedImage bannerImage = EncodedImage.createEncodedImage(StreamUtils.readRemoteStream(imageURL).getBytes(), 0, -1);
		ad.setImage(bannerImage);

		return ad;
	}

	protected TextAd parseTextAd(Document doc) throws Exception {
		TextAd ad = new TextAd();
		parseCommonAdValues(ad, doc);

		//parse the TEXT of the banner
		ad.setText(XML.getTextFromNode(doc.getElementsByTagName("adtext").item(0)));

		return ad;
	}

	protected boolean shouldParseAsImageAd(Document doc) throws Exception {
		Node node = doc.getElementsByTagName("ad").item(0);
		String adType = XML.getAttributeValue(node, "type");

		return adType.toLowerCase().compareTo("img") == 0;

	}

	/**
	 * Parse common values from the responce like the:
	 *  - tracking pixels/beacons
	 *  - the click URL
	 *
	 * @param ad
	 * @param doc
	 */
	private void parseCommonAdValues(Ad ad, Document doc) {
		// CLICK URL
		ad.setClickURL(XML.getAttributeValue(doc.getElementsByTagName("action").item(0), "target"));

		//TRACKING PIXEL URLs = the BEACONs as they are named in SMAATO network
		NodeList nl = doc.getElementsByTagName("beacon");
		for (int i = 0; i < nl.getLength(); i++) {
			String trackingPx = XML.getTextFromNode(nl.item(i));
			ad.addTrackingPixelURL(trackingPx);
		}
	}

}

Please let me know in the comments if you find this useful!

Preface

At some point in the past I was trying to monetize free application on BlackBerry and since most of the advertising networks that actually work with BlackBerry don't have an SDK, I had to work with their WEB APIs directly.

This did not work for me at all (with minor exception for Quattro Wireless). Maybe this was personal to me, I don't know, probably not. The monetization through advertising on BlackBerry is unfortunately does not work. I am talking from a lot of experience supporting free applications with ads on Android and iOS.

I've spent a lot of time looking for advertising agencies working with BlackBerry and then signing up and then getting the necessary information from them and then working through their documentation to implement a custom web client to pull the info from their web APIs - all this to learn once again that the revenue I could get is a joke.

Quattro Wireless

Advertising with Quattro Wireless actually turned out to be the best BlackBerry advertising experience I had so far... As long as it lasted (not very long unfortunately).

Starting Sept. 30 1020 Quattro is iAd exclusive and stopped serving any ads to BlackBerry. Since I do not ever trust what I can get from advertising service, when the service stopped, the application was not impacted but other developers that were less careful (something you should not ever do though) might have had some problems.

NexAge

NexAge left me with the worst impression of all. First these guys at NexAge took a hell lot of my time with manual sign up and many  many phone calls I had to have with them to get anywhere. It looked very suspicious to me that they can't implement an automatic sign up and to actually sign up for the service you had to talk to them over the phone. I should have known better at this point. Coupled with their awfully looking website, this fact should have stopped me from moving forward with them and saving a lot of my time and frustration. But since there were not many options at a time, I decided to try them out. The interesting part is that when I voiced my concerns about the manual sign-up to some higher up developer relations manager, he told me "We are the BMW of advertising, we give you a personalized and best service out there"... yeah... right...  After signing up with them and integrating their code (there were a few hiccups on the way but nothing deal breaking) I started to serve their advertisements and the results were - well... simply put horrible. When I talked to them they were not really helpful and they kept saying "we will investigate and will try to optimize". I wonder if they are optimizing the ads manually as the sign-up. After couple weeks of my pilot with them, during which I wrote them and called them to complain about the ridiculousness of the revenue with not much results (they mostly simply ignored me) I shut them down and decided to forget about them and about the partial revenue I have generated in these couple weeks for the sake of my own mental health. I have to point out that they did have an SDK - a ridiculously large SDK, so I decided that I can not add an SDK which is almost as much as half of my whole application size, especially while I can implement the same thing with couple classes.

SMAATO

With SMAATO everything was much simpler, but the revenue was as bad if not worse as with NexAge. After trying them for some time and not seeing any improvement, I gave up on them.

MillennialMedia

With Millenial, it was more complicated. At some point I was simply furious with them and if there were any other options to advertise on BlackBerry, I would definitely avoid them, but since there aren't I still work with them in a very limited capacity. I was so so sorry to learn that they are going to handle the technology side of RIM's advertising service. The reason is simple - they are SUPER unorganized. And coupled with RIM, they just doubled the amount of mess. For example, they say you are ready to serve ads and you are fully operational and you publish your app and then you see some super weird stats and you ask them and they tell you some BS and then you wait a little more and then you ask again and then a few weeks later they say - "Ohh the ads you were serving are test ads". To make things even worse - this happened to me more than once.

Summary

In summary, unfortunately, if you want to monetize your free application on BlackBerry - you better prepare yourself for the worst. I was hoping the official RIM advertising service will solve this problem, but unfortunately it didn't. It's a complete mess, as I mentioned and I don't think they are going to sort this out any time soon. If you are a developer that knows of a better service - please let me know, I would appreciate it. Hopefully my experience also helps you to save some of your time. If you are eager to try anyway, I plan to post the custom integration code that I wrote here so that you (again) can save some time reusing this code.

I have been to a presentation by Harry Heyman - the first non founder employee of Foursquare. He presented on the Foursquare's usage of MongoDB, a rapidly gaining popularity NoSQL database. Quickly - the idea behind NoSQL database that it is not based on a relational model as most traditional databases but rather on a document model.

Harry had many interesting things but the one thing that was especially interesting to me was that they basically run all the DB operations of FourSquare in RAM! Yes that's right. They employ the extra large memory instances on EC2, running with 64GB or RAM and they shard whenever they start approaching certain limits of RAM.

The outages they had in early October were because they hit the memory limit and the databases started to page in and out the data from the hard drives to the memory and back and this quickly brought them down.

He mentioned that this is however not Mongo's fault and this would have happened with any database they would have used. Which is interesting by itself because it means that the way the application is engineered - the requirements is that all the data that they operate has to be in RAM for the application to be fast enough. I bet there is a lot of smart engineers working at Foursquare and there has to be a good reason for that and in fact Harry mentioned that they basically have to have almost all the checkins available in memory due to the nature of the queries they have to perform on the data.  He also mentioned that when (this seemed more of a "when" rather than "if") they move to dedicated servers with solid state drives, they should not have that limit anymore because then the hard drives (SSDs) in that case will be fast enough.

For me all this was very surprising and I am still a little puzzled about it and wonder whether there is/was no way of engineering the application so that they don't have to keep pretty much all their data (about 5 million users and 160 million checkins at the moment), all in RAM.

Slides of the presentation

UPDATE (Dec 15 2010) a few related links:

Official blog from Foursquare

Interesting discussion at YCombinator hacker news

Some notes of Alex Popescu from NoSQL blog

UPDATE (Dec 17 2010) - 10gen posted the video of the meetup

MongoDB and Foursquare video

The foursquare and MongoDB logos are properties of their respected owners.