#420. Mastering the Art of Lens Testing (a Free Guide)

By pascaljappy | News

Oct 23

If you’re a male photographer, you probably suffer from GAS. Our lady colleagues somehow seem far more capable of focusing on the artistic side of things but we guys place an unhealthy amount of attention on reviews, tests and, worst of all, what other people think of the gear we use.

 

Gear dreams

Gear dreams

 

On the plus side, any new gear announcement has us wagging our tails like puppies waiting for a bone.

But, more often than not, this gear acquisition syndrome is a source of tension. Tension among photographers as the ridiculously angry flame wars on gear forums prove day after day. And inner tension because we realise that new gear is often a proxy for talent which mostly leads to disappointment.

Sure, expert photographers know when a new piece of kit is going to suit their needs and enhance their already high abilities. But most togs just glance at technical features hoping they will replace the long years of introspection, self-criticism and hard work that photography, as any other art form, requires to attain proficiency.

 

Sad

Sad

 

I came to realise this the hard way during my early digital years. Coming out of a love affair with a Mamiya 7 + 65mm lens, I took the plunge and sold all my film + lab gear (Mamiya, Linhof, Schneider …) to finance a Canon DSLR and two L zooms covering 28 to 200mm.

The best (my) money could by, at a time where more and more good photographs were happening to me (still mostly through unprovoked chance but at a more optimistic rate than ever before), would surely only lead to a deluge of works of genius. I probably wondered what the MoMA and Gugg would do with the lesser stuff that would have to leave to make way for my ineffable brilliance.

Well, it was a horrible two years during which I don’t think I shot a single good photograph.

The situation was inexplicable (not to mention heartbreaking), given how much I’d read to choose my optics, the tremendous reviews my Canon was receiving in the hands of Michael Reichman and other luminaries of the craft and the undeniable prodigy nascent inside my mind. It was only years later that I understood how utterly useless I was with zoom lenses at the time.

But at least, I realised. And took action.

 

Hope ?

Hope ?

 

Others aren’t so lucky.

An acquaintance of mine lost his marriage to photo gear. After obsessively engulfing the family’s earnings into white and gold zooms, 10 fps bodies with battery grips and filter systems I couldn’t wrap my brain around, he’d show me his photographs that, even at my early stage of not-so-good expertise, had me scratching my head for clues of flair. It ended predictably. Expensive separation, self-loathing as a free bonus.

 

7 reasons lens reviews make you a bad photographer

The horrible truth about all this is that so many people benefit from your confusion. We have been lured into thinking that lens design and testing is an exact science. It isn’t. It all rests on ergonomic choices and aesthetic choice, personal choices.

 

Alone

Alone

 

And here are 7 reasons why lab lens tests are less than helpful when you are trying to become a better photographer:

  • Aiming a lens at tests charts is quick and easy. There is no certification required. Results are uniform and do not take the rendering of the lens into account.
  • Most measurements are made at about 1m50, simply out of convenience, a distance for which the lens was not necessarily optimised and where it may perform very differently to how you want to use it.
  • No real-life test scene in the world is interesting from f/1.4 to f/22.ย  An aperture series of that sort is simply missing the point and misleading you.
  • Test gear of today’s very high quality lenses is incredibly expensive (roughly $200 000 for a serious MTF rig). How many reviewers have made that investment ?
  • Photos of brick walls are an amazingly accurate test of quality for lenses that will photograph brick walls. If that’s your niche, you are so in for a treat.

Now I’m not implying all lens reviewers are quacks, far from it (some are doing an excellent job). But not all of them give their verdict the thought it would require. And, far more importantly, lets continue with the last 2 reasons:

  • The link between lab reviewer and artist is tenuous at best. When you see the sample pics on review websites, do you want to hang them on your wall?
  • When was the last time you heard Salgado, Bresson, Kenna, … mention the words “test chart”?

 

Tangled

Tangled

 

In the automotive industry, VW are facing the crisis of their lifetime after 11 million vehicles were sold with onboard computers that cheated on pollution tests. How many lab-testing journalists picked-up on that since 2007? Well, the situation is similar in the photo industry, with the one caveat that except for Zeiss and Leica, no manufacturer I know of even bothers testing the MTF of its lenses and simply prints an ideal-world calculated version.

In Oberkochen, Dr Nasse, who plays a major role in designing new lenses for Zeiss, showed me a chart depicting the performance of an OTUS just 10 microns away from the ideal plane of focus. The MTF measured on that plane just 0.01mm away from the ideal plane, was a huge variation from the MTF published for the lens. The higher the quality of the test, the tougher it becomes to test it adequately.

So, many of the curves published on the web are simply crude approximations of reality (explaining why not all of them agree) and you are far better off making decisions for yourself by testing lenses in the field using a very simple and repeatable process.

 

Seeing the light, at last

Seeing the light, at last

 

Over the years, I came to realise how useless the lens reviews I could find in magazines were to me. Charts with numbers, though eloquent to my former self, started to lose any form of meaning or drive. Even though the vastly useless 0 – 60 times of supercars still conveyed the feeling of adrenaline rushing through your spine as you kick the loud pedal, sharpness histograms got me as aroused as a cash-flow chart for a cast iron foundry in a country I’ve never heard of.

What did get my juices going were the photographs I saw in galleries, magazines and, more recently, on the web, created by people who not once mentioned reviews but openly declared their love for a specific lens at certain apertures.

At that point, I understood the only way to buy lenses was to test them for myself. Not the happy-go-lucky photographs of whatever lied in front of me, but a repeatable series of photographs that told me how the lens would react at various apertures, in various lighting scenarios, close up or at infinity …

I started reading about lens aberrations and associating these to corresponding shooting scenarios and image degradations (or enhancements) until I was able to grab a lens for 20 minutes take 20 to 30 deliberate shots and translate the results really easily in a series of objective points relative to vignetting, chromatic aberration, distortion, astigmatism … And, more importantly, to use these test shots to predict how well the lens would cope with the situations that correspond to my personal style and desires.

 

DSC00668

Radiating light

 

Ever since, I’ve been taking care of my own tests and have never again been disappointed by a lens. I have bought cheap lenses this way, and expensive lenses. All have been stop on for me and the cheapoes always satisfied me just as much as the pricier ones. I know many other photographers have done the same for themselves, and all it takes some of them is a quick look at MTF curves and the optical formula to learn a great deal about the usefulness of the lens for their specific use.

In doing so, they free themselves of the opinions of others and spend far more time creating gorgeous photographs that stressing over the semi scientific tests found online and in mags.

But I still believe test photographs in controlled situations are the most informative way of making a good purchase decision. You’d be surprise how many non-prestige lenses are excellent performers. Their usual downfall is a lack of universality, which places them at a huge disadvantage when facing the non-discriminate studio test scene or test chart.

 

DSC05448-3

The sky’s the limit

 

This is why I created this small course entitled “Mastering the Art of Lens Testing”, to add to the list of DearSusan resources. It is absolutely free and will guide you through a simple set of steps to form your own opinion in a very short time. As you will see when you use it, this is also a lot more fun than looking at a chart if you have any real interest in photography.ย I’ll take a walk in the hills over a stuffy basement any day of the week ๐Ÿ˜‰ ๐Ÿ˜‰

It consists of 8 pages, each describing a specific aberration (or set of related aberrations) and containing :

  • A brief explanation of the reason why the aberration exists in a lens, for general background culture
  • A description of what it does to the image
  • How to test for it in a couple of shots in the field
  • How to fix it in post-processing, so as to let you decide for yourself whether it is a serious issue for you or not.

On each page, I have included some photographs I used to test actual lenses that were sent out to me by manufacturers or that I bought for myself over the years. Most of the material contained applies to smartphone cameras, compact cameras, bridge cameras, DSLR cameras and mirrorless cameras alike.

I recently used this method to test the new Loxia 21 in 20 minutes. The review was found on FredMiranda, where Fred himself commented : “This is likely the best review of this lens we will see”. So it works ๐Ÿ™‚

 

DSC05407-Pano

Free lunch

 

Now, I don’t want to turn this blog into a gear forum with all the pitfalls and hatred associated to those. So this is a mail-only course. If you’re interested in reading this, just fill in the small form below and you’ll be testing your first lens in less than 30 minutes from now ๐Ÿ™‚

You will receive a new link every two days to allow for time to apply the ideas in the previous page before you dig into the next. In no time, you’ll be able to enjoy the process and reach informed conclusions about what works for you and what doesn’t. Give it a go a let me know what you think ๐Ÿ™‚

 

 

IMPORTANT: A first message will be sent to you within minutes. If you do not see it in your inbox, please check your spam folder and theย  social, promotion and forum tabs if you are using gmail. Note that we willย never share your email address.

 

Smooth

Smooth

Happy testing !

 


 

  • Ed Haskell says:

    Is this the ‘small form below’ to which you make reference?

    • pascaljappy says:

      Hi Ed,

      the form is the red one with “Mastering the Art of Lens Testing” as title and Free Course in black. It has 3 fields (email, first name, last name).

      Thanks,
      Pascal

  • Jimm says:

    I relate to this strongly. How many times I have tried in shops and shows that prove total lifeless but have good reviews. Thank you.

  • Yeahhh says:

    Why not just posting a link to a free pdf? Or are you testing this course thing?

    Anyway, I subscripted. One of my favorite lens is one nobody talks about (e.g. Tamron). And sometimes I don’t know how to handle super-hyped ones (e.g. Sigma). But when I can’t trust reviews (I always found DS reviews suspicious ๐Ÿ˜‰ ), how can I judge about a lens? Can’t any good photographer make outstanding images with any gear? And how can I try lenses myself? Buy all new releases on Amazon and send them back within 30 days? That would be a pretty bad customer and would take a lot of time but photographing.

    No, I need guidance of what might be interesting and fitting to me – and what not. That’s why I read DS – and some very few others with a similar mindset. When I read about something interesting and maybe get some confirmation from another side, I might test it out myself by taking shots I normally do. But recently it happens that I read about some interesting attributes, did some more research and ended up with completely different gear. And of course, with most reviews it’s like “Aha, interesting …” – and that’s it. I simply don’t have the money, time and energy to check everything out that seams interesting. Everybody has its own agenda. And I just can’t go out and buy every Zeiss lens available – and I don’t need it.

    Btw: You said MTF charts are calculated or fake and non-sense in general – and later say with a quick look at the MTF chart and optical construction you’re able to have an idea of what look this lens creates. How does this go together?

    • pascaljappy says:

      Hi there,

      my experience in marketing has taught me that few people read long PDFs, even fewer subscribe for them ;). And I want people to read through all of the info. So instalments is the way to go. Plus the pages are good for SEO. That’s for the marketing side of things.

      You don’t have to buy and send back all the lenses in the catalog. But when you do buy, it’s important to be able to assess whether it’s up to your hopes.

      And yes, a good photographer can make great photographs with inferior gear. But he’ll always prefer gear he/she enjoys. And testing is just one part of choosing gear.

      Some manufacturers test their lenses and print MTF curves based on these results. Some photographers read a lot of information into these. But other curves are just fantasy doodling and not much use to anyone ๐Ÿ˜‰

      Hope this clears your questions and that you find the guide useful.

      Cheers,
      Pascal

      • Yeahhh says:

        Will your course cover how to detect interesting lenses based on information found online? I mean, that is the tricky part of why we read reviews, right. Should I buy/rent this lens and invest my precious time in testing it? Do I miss the greatest thing in photography? Or can I move on? You said, you (and others) can read MTF charts and construction descriptions. Can you teach us on how to read it and how to apply it to our own style?

        • pascaljappy says:

          Hello again. I’m not sure about information found online. I have have found this generally unreliable, which is why I’m trying to give readers a vocabulary to describe lenses without having to rely on the reviews of others. Regarding reading MTF curves, that’s part of the next series of articles, I promise.

  • Mikkel Madsen says:

    I’d like to read your lens test guide.
    Kind regards
    Mikkel

  • J.S.park says:

    I’interested in yourโ€œMastering the Art of Lens Testingโ€.
    Thank in advance.

  • Klaus says:

    Hi,

    I’m interested in your lens testing guide. Unfortunately I can’t find the form you mentioned above anywhere on this page.

    Thanks
    Klaus

  • Kim Howe says:

    I enjoyed your first lesson. Looking forward to the rest ๐Ÿ™‚

  • . says:

    Minolta 35/1.4… horrid on the testing field. And yet… owning the FE35/1.4, Sig ART 35/1.4, Samyang 35/1.4… I still reach for the old Minolta more than all others combined. Perhaps a wise sage tog can explain why.

    • pascaljappy says:

      Well, it seems you are the sage. Seeing beyong aberrations and into the actual rendering is what all artists do. This course is a first step towards this. Understanding the aberrations and how they affect the image. Have fun ๐Ÿ™‚

  • James Backer says:

    Where is the ‘form below’?? I want to sign up but see nothing at the end of the article or on this entire page??

  • James Backer says:

    Oops, I did not scroll down far enough to see the ‘sign up’ check box. Sorry…

  • Peter Kelly says:

    There is some truth in what you say (MTF charts and clinical lens tests are an appalling waste for most people), but you shatter the strength of your position by either being careless or deceitful.

    First, I question what quality Canon lenses you ‘bought’, as the expected range would be 24-200 (usually 24-70 f2.8 and 70-200 f2.8). Do Canon make an ‘L’ 28-75?

    However, stranger still that you quote as having owned a whole list of extremely high end film gear, that generally need a prodigious knowledge of photography to get good results, yet want us to believe it took more than two years of taking nothing but rubbish with the Canon equipment? Sorry, but I simply refuse to accept that.

    By all means explain why people should put aside all the facts and figures, take a step back and simply look at their pictures, but please don’t make up stories to pretend this has been some ‘Damascene revelation’ which you need to preach to the great unwashed.

    • pascaljappy says:

      Hi Peter,

      careless or deceitful ? Neither. Surely you can find a more positive proposition.

      My Canon lenses were 28-70/2.8 and 70-200/2.8. But the lenses aren’t the issue, I simply wasn’t happy with a heavy zoom concept, that’s all.
      Accept or refuse what you want. Switching from a Mamiya 7 rangefinder and 2 primes to a digital Canon SLR with zooms was a major disruption for me and an unpleasant place to be.

      Though I find the Damascene conversion metaphor quite apt, I also feel your referring to our readers (generally very educated in photography) as the great unwashed quite insulting.

      Cheers,
      Pascal

      • Peter Kelly says:

        There you go again, with a strange reply that doesn’t make complete sense (as well as misrepresenting what I have written).

        I’m not disagreeing with your basic premise that lenses are far more than the sum of MTF graphs and test sheets. However, if you want people to appreciate your reasoning you need to find a better way to express yourself. Your article reads as though you wish to appear highly experienced, yet are unable to properly use good equipment. The two positions are entirely contradictory.

        I’m not sure how you ended up with a Canon 28-70, as that was not designed for digital and superceded quite some years ago.

        With specific regard to your reply to me, I thought the whole article is about lenses, but you now say ‘lenses aren’t the issue’! That is very silly indeed and fully bears out my interpretation.

        Furthermore, I didn’t call the readers ‘the great unwashed’ at all. I merely suggested that the tone of your piece was written as though you were preaching in that manner; the insult is yours, not mine.

        • pascaljappy says:

          OK, we could do this all day, but it’s much more fun to write tutorials that help people than to respond to blind criticism. The 28-70, in the days of the Canon D30 was all there was. And no, I wasn’t very good with a zoom lens at the time (nor do I enjoy them today) or at digital. However, it’s a fact of life than in 13 years, people who try hard get better at things. That’s what I’ve done and that’s what I’m sharing. Don’t like it, no biggie. End of dicussion.

  • Alonso says:

    Wow. This completely changes things for me. Looking forward to this course. Thank you.

    • pascaljappy says:

      Well, that’s pressure on our shoulders ๐Ÿ˜‰ Thank you for the comment. Please let us know if you like the course and if you’d like things added. Have fun !

  • Sean Quigley says:

    This reminds me of my hi-fi days when the world abandoned lab tests and the men with golden ears were born, oh those were the great days of hi-fi, sadly gone forever.

    How a lens render’s is king, that’s why so many of us, from the world of hi=fi are still photographers.

    Great article.

    • pascaljappy says:

      Thanks Sean, it’s amazing how many HiFi nuts are also photography enthusiasts. Lenses are really like HiFi systems in that they can test beautifully and feel dull or vice versa. A new Class D amp, a Mosfet, a 300B push-pull will all sound different and it’s good to be able to put words on what it is that we like or dislike and on what type of music, right? Cheers, Pascal

  • Condell Maurice says:

    Just one more thing that has not been discussed in the digital age with enough due care.

  • Jay Taylor says:

    This guide sounds like a real valuable piece of information especially for those of us who like to tinker (aka lens addiction ) with several pieces of glass. Please send the course to me.

    Thank you,
    Jay

  • >