Building a Custom Keyboard for iOS

Coming soon….

AVPlayer In Depth

Performance

AVPlayer has some nice built-in optimizations that would be a pain to code on your own. I should know because I tried. I did succeed in making multiple simultaneous requests for different byte ranges, piecing the file back together and handing it over to an audio player while continuing to download the rest of the file. However, AVPlayer goes further than that and seems to optimize based on the phone’s network connection as well. Below you will see Request and Response Headers that I pulled out of Fiddler for both WiFi and 3G. You’ll see that over 3G AVPlayer makes several more, and smaller, requests than over WiFi. It’s possible that iOS is even going as far as testing the network speed and not necessarily the type of connection to determine how to efficiently download the file.

Over WiFi this is the first Request/Response:

GET http://some-URL-requesting-a-music-file
User-Agent: AppleCoreMedia/1.0.0.9B176 (iPhone; U; CPU OS 5_1 like Mac OS X; en_us)
Accept: */*
Range: bytes=0-1
Accept-Encoding: identity
X-Playback-Session-Id: D4C23A38-2084-4154-ABAD-930E59EC90D5
Connection: keep-alive

HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Content-Type: audio/mp4
Content-Range: bytes 0-1/2229090
Content-Length: 2
Connection: keep-alive

2 more Requests went out after this. Each one only differing in the Range/Length values:

Request:
Range: bytes=0-2229089
Response:
Content-Range: bytes 0-2229089/2229090
Content-Length: 2229090

Request:
Range: bytes=166896-2229089
Response:
Content-Range: bytes 166896-2229089/2229090
Content-Length: 2062194

Over 3G this is the first Request/Response:

GET http://some-URL-requesting-a-music-file
User-Agent: AppleCoreMedia/1.0.0.9B176 (iPhone; U; CPU OS 5_1 like Mac OS X; en_us)
Accept: */*
Range: bytes=0-1
Accept-Encoding: identity
X-Playback-Session-Id: 93997E07-A405-4473-BB7D-6BEDE7F95C1E
Connection: keep-alive

HTTP/1.1 206 Partial Content
Accept-Ranges: bytes
Content-Type: audio/mp4
Content-Range: bytes 0-1/1728716
Content-Length: 2
Connection: keep-alive

5 more Requests went out after this. Each one only differing in the Range/Length values:

Request:
Range: bytes=0-1728715
Response:
Content-Range: bytes 0-1728715/1728716
Content-Length: 1728716

Request:
Range: bytes=16384-1728715
Response:
Content-Range: bytes 16384-1728715/1728716
Content-Length: 1712332

Request:
Range: bytes=1344-8191
Response:
Content-Range: bytes 1344-8191/1728716
Content-Length: 6848

Request:
Range: bytes=8192-16383
Response:
Content-Range: bytes 8192-16383/1728716
Content-Length: 8192

Request:
Range: bytes=304683-1728715
Response:
Content-Range: bytes 304683-1728715/1728716
Content-Length: 1424033

Properties

Using observers for the AVPlayer status and rate and the AVPlayerItem status yields the results below. What’s strange is that the rate is the first thing to change. This happens before either status is set to ReadyToPlay. Furthermore, the rate change observer fires twice. The second time it seems to fire in conjunction with the AVPlayer status change. However, the AVPlayer status change happens before the AVPlayerItem status change and before any time ranges have been loaded. The final observer to fire is the AVPlayerItem status change and that seems to be the only one you can rely on. At this point everything seems to be saying it’s ready and we have time ranges loaded. Shortly after this happens, the music does indeed start playing.

[player addObserver:self forKeyPath:@”status” options:0 context:nil];
[player addObserver:self forKeyPath:@”rate” options:0 context:nil];
[player.currentItem addObserver:self forKeyPath:@”status” options:0 context:nil];

2012-05-07 14:29:07.814  —–start rate change—–
2012-05-07 14:29:07.817  rate: 1.0 at: 2012-05-07 18:29:07
2012-05-07 14:29:07.818  avplayer status: 0
2012-05-07 14:29:07.819  avPlayerItem status: 0
2012-05-07 14:29:07.823  likelyToKeepUp: 0
2012-05-07 14:29:07.825  —–end rate change—–

2012-05-07 14:29:10.377  —–start rate change—–
2012-05-07 14:29:10.414  rate: 1.0 at: 2012-05-07 18:29:10
2012-05-07 14:29:10.416  avplayer status: 1
2012-05-07 14:29:10.418  avPlayerItem status: 0
2012-05-07 14:29:10.421  likelyToKeepUp: 0
2012-05-07 14:29:10.444  loadedTimeRanges: 0.0, 0.0
2012-05-07 14:29:10.447  —–end rate change—–

2012-05-07 14:29:10.451  —–start player status change—–
2012-05-07 14:29:10.452  AVPlayerStatusReadyToPlay: 2012-05-07 18:29:10
2012-05-07 14:29:10.453  avplayer status: 1
2012-05-07 14:29:10.457  avPlayerItem status: 0
2012-05-07 14:29:10.459  likelyToKeepUp: 0
2012-05-07 14:29:10.470  loadedTimeRanges: 0.0, 0.0
2012-05-07 14:29:10.472  —–end player status change—–

2012-05-07 14:29:10.568  —–start item status change—–
2012-05-07 14:29:10.570  AVPlayerItemStatusReadyToPlay: 2012-05-07 18:29:10
2012-05-07 14:29:10.572  avplayer status: 1
2012-05-07 14:29:10.573  avPlayerItem status: 1
2012-05-07 14:29:10.574  likelyToKeepUp: 0
2012-05-07 14:29:10.578  loadedTimeRanges: 0.0, 22.659773
2012-05-07 14:29:10.599  —–end item status change—–

While watching the traffic in Fiddler and watching the corresponding log files I was able to confirm that the loadedTimeRanges property of AVPlayerItem is the most accurate and trustworthy source of what is really going on with the downloading of your song. AVPlayerItem’s playbackBufferEmpty and playbackBufferFull, at least in my experience, are worthless properties that never change. playbackBufferEmpty always reports true and playbackBufferFull always reports false. If somebody has had a different experience with these properties then please let me know. AVPlayerItem’s playbackLikelyToKeepUp property does change as the file gets downloaded so this property could help you determine if your playback is happening faster than your download in which case the user may end up hearing skips or dead air while listening. It’s up to you to determine how to handle that situation.

Sniffing 3G Data

I always assumed there was some sneaky way to watch the network traffic from my iPhone over a 3G network but I never gave it a shot until today. As it turns out, it’s pretty easy.

I used a VPN connection to make this work. I don’t know of a way to setup a proxy just for your 3G connection but if there is a way then any computer with a public IP Address would work.

I used Fiddler to watch the traffic but any network monitor will work. Here’s a good tutorial on setting up Fiddler. IMPORTANT: You only need to do steps 1 and 2 in this tutorial and then make sure to RESTART FIDDLER!

1. Connect your iPhone to a VPN (iOS: Setting up VPN)
2. On your iPhone, go to Settings | General | Network | VPN and click the blue arrow next to the VPN connection you are using.
3. Now you should be on a screen titled “Add Configuration”. Scroll to the bottom and in the “Proxy” section select “Manual”.
4. For the Server, enter the IP Address of the computer you wish to send the data through (ie. 10.2.1.100).
5. For the Port, use 8888 (assuming you’ve done the Fiddler tutorial listed above)
6. Authentication, in my case, is set to Off.
7. Save those settings and open Safari or any app and start sniffing!