☴ ANALYSIS: iPhone 7 binaural videography

iPhone 7 not available in blue.

There's a saying in the film industry: "nobody gives a f5ck about sound". Although that's not strictly true, it is one of the first areas to be compromised in a tight spot during production.

Apple does give a f5ck about sound having employed the likes of Tom Holman to consult on audio direction.

Whilst I believe the iPhone 7 lack of headphone jack is no big deal considering a wired connection remains possible with an adapter, and everything else like dual lenses, stereo speakers, and fast f1.8 lenses is playing catch-up with Android and Windows 10 Mobile Lumia, there remains a glaring omission in the contemporary iPhone spec sheet.

After nearly 10 years the iPhone does not record binaural video (I say binaural video because saying stereo video could mean a stereoscopic 3D effect, but I mean the two-channel sound with video).

The iPhone certainly has enough microphones to have a binaural recording capability. From what I've seen of mic positioning there has been no major changes with iPhone 7 - it has three - one on the front, one on the back, and one on the bottom edge.

This is precisely the same mic positioning as the HTC 10 which does record binaural video. So this is a choice being made by Apple to not record binaural video, not a technical limitation.

Now it might be that Apple are waiting for another design iteration to "do it right" (the HTC mic positioning records mildly biased stereo because left and right channels are not capturing vibrating air molecules equally and it is applying algorithmic compensation and noise cancellation DSP which increases the noise floor with digital uglies in certain scenarios).

Tom Holman's initials form THX (Tom Holman Crossover).

But when? It's been almost 10 years since iPhone dropped and we are in a vibrantly sounding 21st century that would be lovely to capture binaurally with all that hosing of iPhone video.

Clearly, binaural video recording is not important to corporate Apple since Sony Xperia have been offering high end models with binaural video recording for years, and very good it is too. Again it is clearly a choice by Apple not to offer the function because they see it as not important enough to make their list of priorities.

But here's the thing. If you're someone who appreciates sound and making your own recordings, once you've experienced the depth and breadth of decent binaural recording you just don't want to go back to monaural.

There are third party mics that record stereo (the Shure MV88 comes to mind - very good it is too but ultimately creates a biased stage due to it being on the other end to the lens) but the Airninja Movie Method is about less - less to carry, less to remember, less technicalities, just record your damned movie and be assured of decent results by default with a device straight out-of-the-box.

I appreciate that the telephone is a monaural device but an iPhone is so much more than that. It's a pity that out-of-the-box iPhone only creates monaural content. Proof, if any were needed, that binaural sound remains a niche interest - and therefore, opportunity.

The iPhone 7 is also 'water resistant'. The top end Sony Xperia are essentially waterproof in freshwater shallows (IP68) as demonstrated in the following binaural video which ends up fully submerged underwater.




There's just zero reasons for current iPhone to be the airninja's first choice.

☴ Watchlist: HFR & ASMR

I've noticed an emerging trend amongst progressive filmmakers. Firstly they treat online as their primary global distribution platform (natch) but secondly I am seeing more and more content produced with HFR and ASMR.

HFR has largely been rejected by cinema going audiences, but I believe long-term it will become de-facto standard and 24fps will be regarded by my grandchildren as I regard the quirky looking 18fps footage of a hundred years ago incorrectly played back at 24fps.

Everyone can be a filmmaker using a single device.

It's ironic that as a young filmmaker stuck with 50Hz PAL video, I hated that horrible 'soap opera' look of 50i. I strived for that 'film look' just as some modern filmmakers strive for that 'VHS look'. Madness! I yearned for D1 720x576 but when it was finally affordable it was basically obsolete. I wish I'd discovered Laver's law in my twenties then it may have all made a lot more sense to me.

I remember reading, I forget where - other than in print in the early 90s - that when developing Star Tours the boffins got great reactions from audiences when subjected to HFR (60fps was trialled IIRC) - not interlaced like TV, but progressive, actual frames, like film. It wasn't to be though, probably due to reasons of expense and available bandwidth in existing technology. Wish I could remember what the article was and where I read it!

Reading that, it stuck with me to this day. So HFR came as no surprise to me. I believe more motion data is just as important as UHD and beyond. Kids growing up on console gaming just see a blur when they go to the cinema. In fact, so do I (and I don't even own a console).

Cinema may stick with 24fps (for that extra stop in low-light during acquisition), but it's doubtful if IMAX will. I firmly believe IMAX will supplant contemporary cinema as the 'narrative event experience' because home cinema is more comfortable (and a whole lot cheaper) than going out to the cinema. Audiences are split about 3D at the cinema but frankly that's a big fat red herring that gets rolled out every couple of decades. If you want 3D go see a stage play, they are awesome nowadays.

There's an argument that the dating scene will sustain modern cinema. I call bullshit on that, as Netflix & Chill has been marketed so successfully to the younger generation who have grown up with choice and abundance.

EXT. Star Tours.

No way home cinema will stick with 24fps, it will cater to the console kids who grew up gaming at 60fps and will demand their own normal when voting with their wallets (well, assuming the concept of a wallet survives ...)

HFR, especially at 60fps, solves a lot of problems for the progressive filmmaker.

ASMR hasn't made it into cinemas, mainly because 'true' ASMR required the listener to be wearing headphones. It's unlikely that traditional production technique will ever cater to ASMR. However, again, I believe my grandchildren may be more au fait with it than the general public of today. Whilst it may never be mainstream, I believe its benefits will ripple out just as Hi-Fi has done over the last 40 years or so.

There's no doubt that ASMR is pseudoscience, however I have definitely experienced emotional and 'tingly' responses to sound and ASMR is a good an explanation as any.

Essentially ASMR boils down to two things, of which in my experience only one needs to be present (but both is better).

POV often neglects audio completely.

Firstly, perhaps most importantly, ASMR is positional stereo. This usually means recorded binaurally POV. Not practical for most narratives (great for first-person-shooters that the kids play on their consoles - are you detecting a theme yet?). However, I believe a good stereo sound stage reproduction will suffice (more practical for production of traditional narrative).

Secondly, frequency response. Not necessarily flat, but it needs to be 'clean' rather than 'muddy'. This is because the ear cannot discern the direction of bass frequencies but higher frequencies can be highly positional. Higher frequencies tend to lend more air, and thus feed into those ASMR tingles.

It's totally possible for ASMR to be mono, but it is far more effective in stereo. Stereo allows the ear to pick out positional details due to time differences in the sound wave hitting each ear.

This is why it annoys me that Apple's iPhone cannot record stereo with its three (count 'em!) onboard mics. In every other way, the onboard audio always amazes me except for this glaring omission. Sure, external mics are available (and very decent M/S mics they are too - see video embedded below) but sometimes the onboard mics are all you have - typical during the frugalwave. Onboard stereo mics on an iPhone would no doubt be thought about and positioned correctly in relation to the camera lens (or the corrective/adaptive DSP would be spot on).



So Apple has 800 engineers working on the iPhone's camera that they buy from Sony - so what about the audio-visual?


UPDATE: Yes, I should have mentioned the digital bits that represent latitude and all that good 4:4:4 stuff but I think something akin to Moore's "law" in CCD / CMOS / NEXT TLA will see better and better low-light performance over time - it's going to happen anyway.

Sound Really Is 50% Of Your Movie

In the beginning was the Word, and the Word was God – it’s the big bang that started it all, not the big picture!

The sound wave, not the light wave.

Sooner or later you’ll hear someone say how important sound is – but rather than wave them on and uttering “we’ll fix it in post”, have you stopped to consider what it is that sound actually brings to your movie?

Here’s a quick test. Watch your favourite movie with the sound turned off.

Does it still engage you fully? Honestly? Would you sit so long through the end- credits (or opening title crawl) if it weren’t for the choice of music?

Does that really constitute 50% of the movie? Well, on a purely “count the senses” – vision and hearing – yes, yes it does. Arguably sound can improve the visual, and even make people see things that aren’t there.

Good sound can really help prop up a substandard visual but bad sound is, well, just bad sound. Our eyes provide vision as a dominant sense, our ears provide sound almost as a secondary sense – a sense waiting to alert us to events rather than continuously inform us in the foreground. Vision gives us actual information – “that bloke is closing a door” – sound gives us implication – “I heard the door shut, it must be shut”.

Seeing is believing; hearing is implication. Implication can exercise far more of the imagination than actual seeing (afraid of the dark? things that go bump in the night?) – yet so much sound today is used with direct correlation to the visual, giving little room for imagination amongst the (im)perfectly rendered CGI.

Imagination is possibly the most visual weapon in your arsenal as a moviemaker – and that can be driven by sound moreso than picture.

It’s not all about imagination though. Most directors agree that a performance recorded on-set will be better than any looping session (looping, or ADR, is the re-recording of dialogue in controlled studio booth conditions), although often certain environmental conditions will mean that the production sound recorded on-set or on-location is not useable.

Sound is also positional in a way that vision is not – the cinema screen is immersive, yet sound allows you to hear things off-screen (given at minimum a stereo soundstage to work with). Even with the latest 3D visual technology, you still need glasses and everything is still contained within that rectangle. Sound is reproduction of actual physical waves in the air, broken free and authentic in ways that photons are not (and, of course, vice versa – arguably – headphones are entirely optional!).

We could get into stereo miking techniques here, but I suspect that’s best saved for another post. Suffice to say I’ve been a convert to stereo recording for several years, yes, even for production audio. It gives so much more depth and spatial precision versus the cost of full sound design – cost usually being a major factor in the indie moviemaker’s world (whether that cost be in time spent or money spent).

“Real” sounds obviously work best at “real” locations and not those recorded in the fakery of the staple studio system. In the late sixties, “Easy Rider” and “Midnight Cowboy” promised to liberate the moviemaker from the shackles of over-engineering. Then Star Wars happened and moviemaking was never quite the same again.

What do you think? Is sound under utilised in modern movies? Can indies get one up on contemporary Hollywood by tapping into imagination drawn from intelligent sound rather than absolute CGI that uses sound as a crutch only?

Since originally writing this article in 2012 for OTTfilm (now defunct), it's interesting to see the rise of popular ASMR related content on YouTube vs the cinemas push for 3D glasses and a general overall industry push to 4K imaging, IMAX, and resolutions beyond.

The Switch From iPhone 6 Plus to Sony Xperia Z3+ (iOS to Android)

The airninja switch is complete. As I'm sure you know I am platform agnostic, being primarily interested in the best tools for a given job.

For the last five years I've been an iPhone user, upgrading most years with unlocked devices and running on Three UK PAYG (great unlimited data plans).

Each year I've been waiting for Apple to release an iPhone that is capable of recording stereo sound using its onboard mics when taking video (modern iPhone has three mics). Whilst iPhone mono recordings do sound good, I've always preferred the 'reality' of a stereo sound-stage - and I don't always carry a stereo microphone with me because extra gear is not the way of the airninja.

For various reasons (job redundancy being a huge one) I have been re-evaluating the "things" in my life and when Apple came under the microscope it just didn't hold up. The competition promised to deliver far more value into my life (including the stereo audio video recording).

I bought a refurbished Sony Xperia Z2 for a shade over £200 to see how I'd get on. This phone model is about a year old, records 4K video, has 3GB RAM, expandable storage to 128GB and is made by the company and culture that Steve Jobs himself had great admiration for.

An Apple designer, Shin Nishibori, was asked to create “Sony-like” concepts of phones that carried the name “SONY” and “JONY”. The pictures were created around a year before the iPhone first appeared.

Based on my success with the Z2 (and the release of the Z5) I upgraded to the Z3+ which has an improved camera over the Z2 and a reduced price due to the release of the Z5. They all record stereo audio video and have 4K capability.


I did wonder if Apple's iPhone 6S might offer stereo audio recoding with video, but as far as I know it does not (the iPhone 6 which I owned definitely did not). So, for my airninja purposes, it looks like I made a good choice with the Z3+

Now if you google anything about the Z3+ you'll probably come across reports of how this phone runs hot and overheats. In my experience this is not accurate on the current firmware. Although when taking video it does run warmer than the iPhone ever did. I think it's fair to say that whatever problems may have existed with the Z3+ original firmware has been addressed by Sony in recent updates. HOWEVER a mild annoyance remains with the Z3+ in that the native camera app will report "overheating" and shutdown to cool off. It only does this on looong takes (30 minutes or more continuous) which most normal people rarely do. The workaround is to use an alternative app, such as Google's own camera app (also shoots video).

BTW, I have no compelling reason to ditch OS X or FCPX so I remain an Apple customer.

Competition between vendors is a good thing in general, and who knows I may return to a future iPhone iteration. Yesterday's Apple keynote only confirmed my thoughts on the direction Apple is now taking, and it's not a direction I want to travel when Sony already provides the sufficient destination today.

#DIALW Wind Reel And Print - It's a WRAP!

So technically we wrapped production on Saturday and proceeded to discuss future film projects over a great curry.

It's been a fun week and I look forward to turning in the first finer-cut edit to the director, Seb Hunter. But Monday is a day off, I'll tackle that tomorrow.

The film is already loosely cut and clocks in at around 28 minutes sans titles. But too loose to show anyone, just yet.

Floor 6, Chesil Street MSCP. Park by phone.

So let me reflect on what I've learned on this production -

1. To park by the statue of King Alfred on Broadway in Winchester requires 'coins'.

2. In freezing muddy conditions you can do a lot worse than Vivobarefoot Tracker hiking shoes.

3. In freezing muddy conditions at night you can do a lot worse than Nitecore TM26 at 150 lumens with a runtime of approx 41 hours.

4. If the production unit is highly mobile with added agility then kit choice needs to reflect this. Mobile means why not use mobile. Thus, airninja movie method is validated.

5. Even though we are not using production audio from the iPhone 6 Plus, if we had planned for it, we could have. Audio perspective matches the fixed lens, MEMS microphone DSP helps noise reduction (although it didn't reduce the noise of overhead aviation, it won't do miracles nor should you expect it to).

6. All I want from a video camera app is simple - fix the white balance to a preset, fix the exposure, but leave autofocus roaming (AF with focus pixels works very well in iPhone 6 Plus). Currently the native app can't do this, so the closest I have found to my needs is Kinomatic.

7. Parkas are in fashion this season.

8. I didn't need touch sensitive gloves to operate the iPhone - just a naked hand and a spare pocket to keep it in between takes.

9. I used the ShoulderPod S1 + Manfrotto PIXI with wrist lanyard 95% of the time. The other 5% was on a Manfrotto Compact Action tripod.

10. Digital zoom, you guessed it, sucks. Which means all next wave rebellion against airninja must use digital zoom.


Look out for the final cut of Dark Is A Long Way playing in Winchester cinemas with live musical accompaniment from the Provincials - ETA Easter 2015.

iPhone Strangelove (or, how I learned to stop worrying and move closer)

Today I recce Spinney Hollow along with the director and cast of 'Dark Is A Long Way' (DIALW). I will be taking my shooting kit with a view to get some test footage and just a general dry-run of British winter cold and boggy shooting conditions.

Shooting with iPhone (this is essentially a zero budget shoot, so zero kit hire), I am well aware of its limitations but also of its supreme advantages. If limitations can be turned into advantage, so much the better.

What strikes me about iPhone is the fixed lens and lack of optical zoom. This isn't necessarily a disadvantage on an airninja movie method shoot. If you resist the digital zoom (and I never use it) then the onboard mic gives a good representation of audio perspective and in my experience a high degree of clarity.

Let's talk about the iPhone onboard mic. I mean, recording production audio with any onboard mic is absolute heresy, right? A smartphone mic must be atrocious, right?! Don't be so quick to dismiss.

Firstly, the iPhone has three onboard mics, of the MEMS variety. This allows DSP to cancel out off-axis noise which can really help in reportage and documentary situations.

Specs on the iPhone MEMS mics are hard to come by. They have a relatively high noise floor (current generation around 64dB SNR is some way off desirability) but are generally capable of capturing HiFi 20Hz - 20kHz (how flat is anyone's guess - I suppose that is where the DSP comes in too). To my ear though, it always sounds good given constraints of whatever the recording situation was. Traffic noise isn't going to suddenly disappear but damn it if the person talking isn't more intelligible than when I was actually there listening with my own ears. Amazing!

This is no coincidence. For a few years Apple's audio was directed by Tomlinson Holman who should need no introduction to anyone in the audio field. More recently, Dana Massie who was previously director of DSP at Audience.

It makes sense that a device optimised for phone calls in highly variable environments would also do the best it can in any given video recording situation. High wind remains an achilles heal and can start the DSP overapplying noise reduction until things sound very odd.

Now, admittedly DIALW has been written as almost a silent movie, with very sparse moments of dialogue. In quiet, low wind conditions the iPhone onboard mics do a great job, especially if you are close to the subject. We proved this shooting GAIN (albeit that was shot on iPhone 4S). Although I intend to record any DIALW dialogue with my Apogee MiC 96K (a digital signal path directly to the lightning port), I also intend to use production audio from the iPhone if I consider it good enough. I am confident it will be (famous last words). Then, I can declare MEMS mics an official part of the Airninja Movie Method and stop worrying.

DeerCam1 Revision 2 - optimising iPhone microphone background noise reduction

Field testing DeerCam1 with the Griffin Survivor case revealed an audio anomaly that was attributed to the iPhone's excellent background noise reduction DSP being confused whilst encased in Griffin's excellent dust-resistant military grade protection. Whilst in the case the iPhone would record a 'clicking' or 'ticking' noise in the higher frequencies. Weird! Multiple tests across two devices confirms the Griffin case as the cause.

DeerCam1 / Anker Glaze Case / Shoulderpod S1 / Manfrotto Pixi

With shooting of 'Dark Is A Long Way' due to commence 17th January 2015, a new solution has to be found. The Griffin case had manual cut-outs applied which resolved the issue, but did not satisfy my OCD. If I have to have cut-outs, why not get a case that already has cut-outs?! But also a case with an integrated screen protector which (proven whilst shooting GAIN on 4S) helps continue shooting in bad weather and rain without needing somebody to hold an umbrella over the iPhone (somebody holding an umbrella = additional admin).

Currently field testing DeerCam1 suited up in the Anker Protective Case. This case has a generous cutout around the back camera and microphone and a cut-out for additional mics - this should allow for unfettered DSP whilst recording.

Generous cut-out around back camera and microphone

Port cut-out for side (noise cancellation matrix) microphone