My Web Log Book

Core Data gets batch updates!

This had been a performance pain point with Core Data when you want to update thousands of objects and change a single property. An example given in this article points out a fairly typical use-case for RSS readers. If you want to update thousands of objects and set the read property to YES, then you’d have to load all those objects in memory, perform the change, and then save each object. Well, I hope that’s not true anymore with the new NSBatchUpdateRequest class. Here’s what the docs say about it:
//  Used to request that Core Data do a batch update of data in a persistent store without
//  loading any data into memory. May not be supported by all store types.
//  WARNING:
//  It is up to the developer creating the request to ensure that changes made by the request to
//  the underlying store do not violate any validation rules specified in the model.

Caching audio streamed using AVPlayer

AVPlayer is quite convenient for playing audio/video files from the disk or the network. However, you may want to cache the audio that was played by AVPlayer but it is not straightforward. A good alternative for this used to be the AudioStreamer library written by Matt Gallagher. I’ve been using the library for sometime now, and although it’s a fantastic library, there are some quirks that I’ve had to deal with especially around race conditions that are hard to reproduce and an ‘hwiu’ exception. Nonetheless, this library had served us well. Moving on..

I was looking for solutions to get AVPlayer to play an audio file from the network but also cache the downloaded data so the next time I can play it locally. Unfortunately that’s not a very straightforward thing to do. I found a couple of helpful hints that tapped into the audio processing and re-recorded the stream as it was being played back by the system. This article from venodesigns.net had been tremendously helpful in implementing that. It worked great for the most part but it was unsuited for my purposes because I wanted the entire audio cached without interruptions. If the user seeked anywhere in the audio, my cached audio had gaps in it.

So I started looking around digging through the APIs and came across the resourceLoader object in AVURLAsset. This is actually an amazing API using which you can provide controlled access to a remote audio file to AVPlayer. This works like a local HTTP proxy but without all the hassles.

The most import thing to remember about this resourceLoader is that it will only come into play when AVPlayer does NOT know how to load a resource. It is quite equipped to deal with the “http” protocol so as long as you have http(s) URLs, no callbacks will be made. The trick here is to change the protocol so AVPlayer is forced to defer the loading of the resource to our application. The resource loader has two methods we need to deal with. We’ll store the requests that are still pending in an array called pendingRequests. Then we’ll start off an NSURLConnection to fetch the audio file, and as and when we retrieve more data, we will try to process the pendingRequests, if possible. The two delegate methods of AVAssetResourceLoaderDelegate we have to implement are:

// Called when the resource loader needs data or information about the resource. To take over the resource loading, we'll return YES from this method.
- (BOOL)resourceLoader:(AVAssetResourceLoader *)resourceLoader shouldWaitForLoadingOfRequestedResource:(AVAssetResourceLoadingRequest *)loadingRequest

// Called when a load request is being cancelled.
- (void)resourceLoader:(AVAssetResourceLoader *)resourceLoader didCancelLoadingRequest:(AVAssetResourceLoadingRequest *)loadingRequest

To start off, we’ll create an AVURLAsset object, change the URLs protocol to a custom one, and set the resource loader’s delegate to be our class.

// Change protocol to streaming from http
AVURLAsset *asset = [AVURLAsset URLAssetWithURL:@"streaming://sampleswap.org/mp3/artist/earthling/Chuck-Silva_Ninety-Nine-Percent-320.mp3"];
// Set resource loader on the asset so that we can control the loading process
[asset.resourceLoader setDelegate:self queue:dispatch_get_main_queue()];

// This tracks all pending AVAssetResourceLoadingRequest objects we have not fulfilled yet
self.pendingRequests = [NSMutableArray array];

AVPlayerItem *playerItem = [AVPlayerItem playerItemWithAsset:asset];
self.player = [[AVPlayer alloc] initWithPlayerItem:playerItem];
[playerItem addObserver:self forKeyPath:@"status" options:NSKeyValueObservingOptionNew context:NULL];

The delegate methods are basically responsible for adding and removing a request from the pendingRequests array. When the first request is received and no NSURLConnection exists, we’ll create one and start fetching the audio file.

- (BOOL)resourceLoader:(AVAssetResourceLoader *)resourceLoader shouldWaitForLoadingOfRequestedResource:(AVAssetResourceLoadingRequest *)loadingRequest
{
    if (self.connection == nil)
    {
        NSURL *interceptedURL = [loadingRequest.request URL];
        NSURLComponents *actualURLComponents = [[NSURLComponents alloc] initWithURL:interceptedURL resolvingAgainstBaseURL:NO];
        actualURLComponents.scheme = @"http";

        NSURLRequest *request = [NSURLRequest requestWithURL:[actualURLComponents URL]];
        self.connection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:NO];
        [self.connection setDelegateQueue:[NSOperationQueue mainQueue]];

        [self.connection start];
    }

    [self.pendingRequests addObject:loadingRequest];

    return YES;
}

- (void)resourceLoader:(AVAssetResourceLoader *)resourceLoader didCancelLoadingRequest:(AVAssetResourceLoadingRequest *)loadingRequest
{
    [self.pendingRequests removeObject:loadingRequest];
}

The NSURLConnection delegate callbacks try to process any pendingRequests in all callbacks.

- (void)connection:(NSURLConnection *)connection didReceiveResponse:(NSURLResponse *)response
{
    self.songData = [NSMutableData data];
    self.response = (NSHTTPURLResponse *)response;

    [self processPendingRequests];
}

- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data
{
    [self.songData appendData:data];

    [self processPendingRequests];
}

- (void)connectionDidFinishLoading:(NSURLConnection *)connection
{
    [self processPendingRequests];

    // Done loading, cache the file locally
    NSString *cachedFilePath = [NSTemporaryDirectory() stringByAppendingPathComponent:@"cached.mp3"];
    [self.songData writeToFile:cachedFilePath atomically:YES];
}

The request handling is the crucial part. An AVAssetResourceLoadingRequest has two parts to it - contentInformationRequest, and dataRequest. A contentInformationRequest is a request to identify the content length, content type, and whether the resource supports byte range requests. With byte range requests, AVPlayer can get fancy and apply various optimizations. The first request I’ve seen asks for the first two bytes of data. Here is an implementation of the request handling part:

- (void)processPendingRequests
{
    NSMutableArray *requestsCompleted = [NSMutableArray array];

    for (AVAssetResourceLoadingRequest *loadingRequest in self.pendingRequests)
    {
        [self fillInContentInformation:loadingRequest.contentInformationRequest];

        BOOL didRespondCompletely = [self respondWithDataForRequest:loadingRequest.dataRequest];

        if (didRespondCompletely)
        {
            [requestsCompleted addObject:loadingRequest];

            [loadingRequest finishLoading];
        }
    }

    [self.pendingRequests removeObjectsInArray:requestsCompleted];
}

- (void)fillInContentInformation:(AVAssetResourceLoadingContentInformationRequest *)contentInformationRequest
{
    if (contentInformationRequest == nil || self.response == nil)
    {
        return;
    }

    NSString *mimeType = [self.response MIMEType];
    CFStringRef contentType = UTTypeCreatePreferredIdentifierForTag(kUTTagClassMIMEType, (__bridge CFStringRef)(mimeType), NULL);

    contentInformationRequest.byteRangeAccessSupported = YES;
    contentInformationRequest.contentType = CFBridgingRelease(contentType);
    contentInformationRequest.contentLength = [self.response expectedContentLength];
}

- (BOOL)respondWithDataForRequest:(AVAssetResourceLoadingDataRequest *)dataRequest
{
    long long startOffset = dataRequest.requestedOffset;
    if (dataRequest.currentOffset != 0)
    {
        startOffset = dataRequest.currentOffset;
    }

    // Don't have any data at all for this request
    if (self.songData.length = endOffset;

    return didRespondFully;
}

Link to the full gist.

Simple tip for improving error handling in your Objective-C app

Objective-C prefers error objects in-place of exceptions most of the times. That error will get populated with a valid pointer if something failed, otherwise it will be nil. If you don’t care about the error, you can just pass in nil in almost all APIs. However, if you are working on a large codebase or with a team, or if you want to write a good robust app, you should handle all errors meticulously. Once convention I’ve come up with is to force the callers to pass in a valid pointer instead of NULL. It’s as simple as putting some asserts. Here’s what I do when I want to force the callers to take care of error checking.

Next step is to ensure that the caller actually handles the error or bubbles it up appropriately. That’s a whole different game, but at least we get the ball rolling this way.

- (BOOL)fetchProfilePictureWithError:(NSError * __autoreleasing *)error
{
    NSParameterAssert(error != NULL);
    NSParameterAssert(*error == nil);

    // do the fetch
}

It sounds very heavy-handed but my experience says it’s better to enforce error handling sooner than later and this pattern has helped quite a bit.

Synchronizing around a class

A common technique I’ve been using to synchronize access to class methods is to wrap a piece of code that needs synchronizing with an @synchronized block. For example, say we have an Animal class that has a class method +totalPopulation, I would’ve synchronize it as:
@interface Animal : NSObject

+ (NSUInteger)totalPopulation;

@end


@implementation

+ (NSUInteger)totalPopulation
{
    @synchronized (self) {
        // do something fancy to figure out the
        // total population of all animals
    }
}

@end
The problem starts when there are subclasses of Animal and we call +totalPopulation from Animal, and a subclass in different threads simultaneously. Both methods would be allowed to run at the same time because the lock is held using two different objects.
@interface Dog : Animal
@end

@implementation Dog
@end
So +totalPopulation is no longer thread-safe when we call [Dog totalPopulation] and [Animal totalPopulation]. The fix is to ensure that we hold the lock using a specific object. So in this case instead of using self, we would specify the class object on which we wish to lock.
+ (NSUInteger)totalPopulation
{
    @synchronized ([Animal class]) {
        // much better
    }
}
Lessons learnt: Be as specific as possible when locking methods.

NSCalendar is always looking out for you

Just got this message from calling -[NSCalendar components:fromDate:toDate:options:]

2014-04-16 19:34:12.319 App[6140:3c0b] *** -[__NSCFCalendar components:fromDate:toDate:options:]: fromDate cannot be nil

I mean really, what do you think that operation is supposed to mean with a nil fromDate?

An exception has been avoided for now.

A few of these errors are going to be reported with this complaint, then further violations will simply silently do whatever random thing results from the nil.

Here is the backtrace where this occurred this time (some frames may be missing due to compiler optimizations):

(
	0   CoreFoundation                      0x33e8548f  + 86
	1   App                                 0x00153825  by me
	2   App                                 0x00155ec9  by me
	3   libdispatch.dylib                   0x3bf7411f  + 10
	4   libdispatch.dylib                   0x3bf7399b  + 146
	5   libdispatch.dylib                   0x3bf73895  + 36
	6   libdispatch.dylib                   0x3bf82215  + 192
	7   libdispatch.dylib                   0x3bf823b9  + 84
	8   libsystem_c.dylib                   0x3bfa8a11  + 360
	9   libsystem_c.dylib                   0x3bfa88a4 start_wqthread + 8)

Objective-C humor

I came across a bug in my application today where the -main method was overridden in a subclass but the superclass implementation was not being called. The solution was straightforward though. Thank you [super main].

Protected stuff for Objective-C subclasses

That’s a good name for protected extensions for a subclass that I came across in UIGestureRecognizerSubclass.h
@interface UIGestureRecognizer (ForSubclassEyesOnly)
…

Date parsing performance on iOS (NSDateFormatter vs sqlite)

Many web services choose to return dates in something other than a unix timestamp (unfortunately).

Recently I set out trying to identify performance bottlenecks with a large import operation in our iOS app. Having tweaked most of the variables, I was surprised to find out that date parsing was actually one of the biggest bottlenecks. The date parsing was done using NSDateFormatter using an ISO-8601 formatted date that looked like this:

2013-09-07T23:45:00Z

That looks simple enough. We had the NSDateFormatter’s format setup with a timezone of +0 GMT, and everything was great, expect parsing dates like this was consuming around 20% of the entire import operation. To provide some context, we were testing the import performance by importing roughly 250,000 objects into a SQLite database, and each object had 4 dates associated with it. That meant that we were dealing with a million dates, and parsing a million dates can get expensive.

Almost all of that time was being spent inside NSDateFormatter’s dateFromString: method, so there was not much we could do to optimize things ourselves.

The main goal was to get a unix timestamp from the ISO date. Luckily for us, SQLite is quite good at parsing some ISO-8601 dates and works blazingly fast. Here’s how SQLite can parse the above date:

sqlite> SELECT strftime("%s", "2013-09-07T23:45:00Z");
1378597500

Let’s convert this to use SQLite’s C library.

sqlite3_stmt *statement = NULL;
sqlite3_prepare_v2(db, "SELECT strftime('%s', ?);", -1, &statement, NULL);

sqlite3_bind_text(statement, 1, [dateString UTF8String], -1, SQLITE_STATIC);
sqlite3_step(statement);
sqlite3_int64 interval = sqlite3_column_int64(statement, 0);
NSDate *date = [NSDate dateWithTimeIntervalSince1970:interval];

Looks ugly, but we’re trying to solve a genuine problem here. So, how did this do in a performance run against NSDateFormatter? About 1400% faster. To parse a million randomly generated dates on an iPhone 5 running iOS 7, NSDateFormatter took a whooping 106.27 seconds, while the SQLite version took just 7.02 seconds.

Here’s a link to a gist containing the source code used for this comparison.

If you’re curious as to what SQLite is doing under the hood, checkout the date related code in SQLite at http://www.sqlite.org/src/doc/trunk/src/date.c. It mentions that the conversion algorithms it is using are from a book named Astronomical Algorithms by Jean Meeus.

95% of the performance problems are our assumptions.

—Found this brilliant quote on a Stackoverflow comment. Unfortunately, I can’t find a link to the source.