Archive for the ‘Digital photography’ Category
My Photography Workflow
Since the sale of Phanfare to Carbonite, my workflow has changed a bit and become pretty complicated.
I shoot mostly with a smartphone now and from time to time with my Canon DSLR. That probably makes me like most prosumers.
When I was CEO of Phanfare, I did not keep any deliberate local copy of my photos and videos. I did have photos and videos on lots of computers but not organized. After the sale in 2011, when it became clear that Carbonite was going to let Phanfare languish, I downloaded a full copy of my account to my local hard drive.
I took the local copy of my Phanfare account, about 250GB, and put it in Dropbox. That’s a big directory and I don’t want it mirrored to all my computers, so I now have selective sync turned on most places.
I tried to get into Dropbox Carousel, but never found it very engaging. The sharing is not great – initially there was no way to really view on the web at all. And Dropbox does not do much to auto-organize your collection.
For smartphone photos, I usually share select favorites on Facebook, usually within minutes of shooting. Those photos also get mirrored To Dropbox, since Carousel is on my phone. You can’t beat Facebook for getting the immediate gratification of having your photo seen and acknowledged.
I have iCloud Photo Library activated on my laptop, desktop, iPad, and iPhone and it does a reasonably job of keeping my Apple-ecosystem photos synched. I rarely open the desktkop version unless I need to grab a photo for use in email or iMessage from my home computer. But I do like seeing my whole collection on my iPad.
Turning on iCloud Photo Library quickly used up my 5GB free limit from Apple so I pay them.
When I shoot with my Canon 5D DSLR, I acquire the photos into Adobe Lightroom on my home iMac desktop and then create a dated directory within my Phanfare archive in my Dropbox folder of the form ‘2015-07-15 Geocaching with the Kids.’ Sometimes I share a few on Facebook.
Recently, I discovered Google Photos. Google Photos is much like iCloud Photo library in that it will sync your content across your devices and keep a master copy in the cloud, but much, much better.
Google Photos automatically brings together sets of photos as stories, organizes all your photos by face, and groups them by scene (skiing, boats, golf, etc). They also create videos, panoramas, animated gifs and ask you if you want to add them to your permanent collection. It is the most successful attempt yet of cracking the problem of organizing your long tail of personal photos and videos automatically.
If you shoot with a smartphone, Google Photos has location info for each photo and a timestamp and does a terrific job of creating stories of your photos that include location and dates. They can pretty much figure out when the excursion started and ended.
Google photos is more hit and miss with older media that is not geotagged, like photos taken with DSLRs, but it is clearly the best attempt so far. I liked it so much, I decided to pay Google to store fullsize images for me. I also run Google Photos uploader on my desktop and it watches that Dropbox directory, uploading anything I place there from my DSLR.
To help Google Photos do a better job, I have started to geotag my DSLR photos, which you can do with a tracking app that keeps a time-keyed log of where you have been.
I still get a quarterly DVD archive from Phanfare, although I now add only major events to Phanfare. So at this point, I use Facebook, Google, Dropbox and Phanfare for my photos and videos. I pay the latter three. I have these services arranged in a way that they don’t interfere with each other.
I still love my Phanfare website, with its carefully curated album, custom music and descriptions of each event. But if that site goes away because Carbonite accidentally or deliberately ends the Phanfare service, I will never go to that trouble again. I have the meta data downloaded from Phanfare, so the information won’t be lost, but it will probably never be seen by another person outside myself.
My current prediction is that Google Photos wins for the long tail of our personal media. They are cross platform, free for most users (images below 16 megapixels and video of 1080p or smaller), and seem to be making enormous investments in photo analysis and automatic organization.
Timing Matters
Mark Heinrich and I started Phanfare sometime in 2003. We incorporated in 2004 and eventually sold the company in 2011. Along the way, I learned a few things, but one of the most memorable to me is that timing matters. Not only can you be too late, but you can also be too early.
When we started Phanfare, storage was too expensive to consider storing fullsize originals for free, ad-supported. Collections averaged more than 30GB and our fully burdened cost of storage was at least $1/GB/year. Even Google today has only about $20 of revenue per user per year ($60B/3B people), and they have one of the most lucrative advertising businesses in the world.
Ask most married people what they would take with them if their house was on fire and they would say their wedding album. Ask them the last time they looked at that wedding album, and they often can’t remember.
Excepting the immediate gratification one gets by sharing a photo or video, the amount of time that folks spend looking at the older media in their own collection, on average, is very small, probably under one hour per month.
Netflix costs maybe $9/month and many families use it over 100 hours. It’s rational that a consumer simply won’t pay more than a few cents, at most a dollar/month, to have convenient and permanent access to their photo and video collection. Obviously, some will, but that makes for a niche market.
So consumers won’t pay much, if anything for a permanent online archive. If you want to make the service mainstream, you will need to make it free. But since total page views and audience attention is low, advertising is not going to cover the bills easily. And if you are storing fullsize originals, then we are not, even today, at the point where delivering a phanfare-like service is economical based on advertising.
Today, the two leading mainstream services for taking your whole collection for free are Apple Photos and Google Photos. Google down-samples by default, unless you are willing to pay. Apple offers only a paltry 5GB for free (if you consider buying a $700 phone free).
So if you want to store fullsize originals for consumers and encourage them to store their entire collection, you need to charge them. And since we established that they will only rationally pay maybe a few dollars per year at most, back in 2008, you were not delivering the service. Phanfare was $99/year. At that price, the service had contribution margin at scale, but getting enough subscribers proved difficult.
So what should we have done? Well, to start, we should not have worried about fullsize originals. We clung to that because we were not sure we had not enough differentiation in the down-sampled photo world. But the reality is, if you show a friend a 25 megapixel image of your children versus a 2 megapixel compressed JPEG of the same kids, the friend leaves with the same impression. The emotional impact is the same.
This lesson on the importance of timing, and secondarily on the importance of doing some price-based-costing (deliver a product at the price point the customer will pay) is one that I won’t soon forget. Technology marches on, becoming cheaper and more capable. When Apple decided in 2001 that your computer would be your digital hub and you would sync by-wire your songs from your desktop to your devices, it was not the best solution imaginable, it was the best solution possible at the time.
The Road to Phanfare
Carbonite acquired Phanfare in 2011 and at the time, I could not talk too much about the acquisition. But some time has passed now and the details are not nearly as sensitive. In this post, I don’t want to focus so much on what has happened since the acquisition (not much) but instead, what led me to start the company with my friend Mark Heinrich.
My love for photography started at Stuyvesant High School in the early 1980s. Back then, a friend who had a Canon AE1 taught me the basics of SLR photography. I bought a Pentax K1000 and eventually a Nikon FG. I still remember trading my K1000 for the FG, taking the modest credit that Willoughby’s offered me. I wish I had kept that first camera.
I became photo editor of the Stuyvesant Spectator newspaper during my senior year, 1985. By that point I had a darkroom in our laundry room at my childhood home on Staten Island. I developed all the Tri-X pan for the staff photographers, creating proof sheets. I had modest equipment and a lot of patience.
I never considered photography to be art. Maybe some photos are art, but most simply document life, communicate a moment. For me, photography satisfied my desire to leave something behind, to be more than dust in the wind. I also loved the technical aspects of photography and the gear. The Japanese cameras are beautifully made, finely engineered instruments. They are almost like jewelry.
I took a lot of photos – as many as I could afford to develop. I always had basic equipment. In college at Dartmouth I had only a 50mm lens and one terrible Vivitar zoom that was so poor I could never get myself to mount it. I joined a company at college, Picture This, that took photos at sorority and fraternity formals. I learned how to shoot portraits and groups, use a flash properly and avoid red eye. I learned that at sorority formals, photos without men sell better than those with the men. The men were transitory.
When digital photography started to first take off in grad school at Stanford, I played with a Apple QuickTake 100 we had in our research group. In the late 90s I watched the founding of Ofoto and Shutterfly with great interest. These were ultimately transitional digital photo services – they focused on creating physical prints and objects from digital images, something with little long term appeal.
Many of us in grad school had simple web sites that we maintained on our workstations with our personal digital photos. These websites were the equivalent of the personal photo album of our childhood, the album that was in the living room cabinet. Most of the images that we published to these sites we got from CDs that Kodak gave us when they developed our film. digital cameras were poor quality in the mid 90s and expensive, way outside the budget of a graduate student. Our websites were often static HTML pages generated by Perl scripts. They were protected from prying eyes via an .htaccess file that popped up a password prompt.
After Stanford, in 1999, I founded my first company with my friend Mark Heinrich: Flashbase. I will talk about Flashbase another time, but it had little to do with photography. DoubleClick acquired Flashbase in 2000 and Mark and I finally had some financial security for the first time in our lives. We started buying a lot more digital cameras. I owned one of the first Nikon CoolPix cameras and the Canon D30, D60, and so on. Our digital images were piling up and we were not sure what to do with them.
Keep in mind, this was before facebook, before YouTube. By 2004, Mark and I were interested in doing something in digital photography. We both loved it, and we both had no idea where we should be keeping the long tail of our digital photos. We were certain that keeping them on our local hard drive was the wrong answer, that your hard drive was where photos go to die. Unseen by anybody, the disk would eventually fail. We hypothesized that folks would pay for a high quality archive of their digital photos. And we thought that the internet should be your master copy of your digital photos, not your computer.
The idea of the cloud being the master copy of your digital photos was not popular yet in 2004. Networks were not fast enough and disk was way too expensive. Steve Jobs was pushing the idea of your Mac being your digital hub, centered around iTunes. To make the whole thing workable, we designed a fat network client that would run on your PC and sync your photos to internet, moving small versions fast and fullsize versions in the background. The idea was that you could work with your photos on the local app while the app moved the photos to and from the net in the background.
We were fixated, perhaps wrongly, on preserving fullsize originals. And at that time, there was simply no way to store fullsize originals and have the service be ad-supported and free. Plus, we did not want ads on our personal photo albums. And so we decided that Phanfare needed to be a subscription service.
I will tell the Phanfare story in another post, but in retrospect, we were not wrong about where photography was going but we were way too early. It would take another ten years before Apple would release their Photos app that synchronizes your iPhone roll with the cloud and shows it across your devices. Google Photos is the same idea (but better). These ideas are right in my opinion. You should be able to shoot and the photos should magically get mirrored to all your devices and stored reliably in the cloud. There should be no upload step.
What we were wrong about is that websites of photos at unique URLs would be the way people share photos. People share photos through social media when they want to share with more than a few people and by direct message (email, SMS, google hangouts, etc) when they want to share with a single person.
I learned a lot from the Phanfare experience and I will share some of those things in coming posts.
Phanfare Reaches Profitability
To our customers,
I am delighted to announce that Phanfare has reached profitability. We had our first profitable month ever in June 2010 and will continue to be profitable going forward on both a cash and accrual basis. This is a significant accomplishment for the company and I am incredibly proud of our team for making it happen.
Mark Heinrich and I founded the company in June 2004, six years ago. Our goal was to enable you to preserve your photos in original quality for generations to come. Like all startups, we have had our ups and downs. But we have never wavered in our commitment to you and preserving your photographic assets.
We raised prices in June to $99/year for Premium and $199/year for Pro to fix the economic model of the company and make the company sustainable.
We bet the company on that price increase. If significant numbers of you had quit, we would have had very few alternatives.
It took me a long time to get to the point of being willing to raise prices. I knew that raising prices would slow customer growth (it has), but I also knew that we were quickly running out of cash. My logic went like this: I will raise prices to a level where we will be profitable at our current size and let you choose whether we get to exist or not.
You have chosen to pay more for Phanfare and enable us to continue to serve you. Profit is not the purpose of business, but it is certainly a requirement. Peter Drucker believed the purpose of business is to create a customer, and I agree with that.
We have wonderful plans for Phanfare and we plan to be around a long time. We have visions of connected cameras and ubiquitous access to your photos and videos, all backed by a ultra-reliable service that preserves the fidelity of your images.
Thank you for taking this journey with us. It has hardly begun.
Andrew Erlichson
CEO
The End of Unlimited Data at ATT is Mostly a Good Thing
ATT today announced the end of unlimited data plans for wireless customers. Why did they do it? It was not about revenue. They sat around the table at ATT and asked: What’s the number one issue with customer satisfaction today? And the universal answer is network performance. How do they fix it?
Well, they could add more capacity but that takes a long time and is very expensive. Instead, they decided to ration the scarce resource of the data network by offering tiered plans. When people pay for what they eat, they eat less and more efficiently. I remember my freshman year at Dartmouth we used to have all-you-can-eat-dining at “Full Fare.” In that dining hall, the trays would go to the dish room with food sculptures and uneaten desserts. Meanwhile, in the “A la Carte” dining room next door, where students paid for each item, trays would show up at the dish room with an empty plate and a fork. Really. It’s just human nature.
With tiered pricing, ATT’s network performance is going to improve in high congestion areas. 98% of their customers will see reduced costs and the top 2%, if they want to pay for extra data above 2GB, will get good download speeds on their additional usage. Plus, top 2% customers are no longer the enemy of ATT. They pay a fair rate and can do whatever they want.
It’s also good for consumers because you can now get a cheaper entry level data plan for $15/month, great for kids.
Alas, ATT could not help but throw in a few items that are not customer friendly or rational. I hope they rethink them:
- If you go over the 200MB in the entry level plan, they charge you an additional $15 for the next 200MB, effectively charging you $30 for 400MB when you could have purchased 2GB for $25 if you had planned better. This harkens back to the very consumer unfriendly practice of making consumers guess at their voice usage minutes per month and hitting them with unreasonable overage charges when they guess wrong.
- Tethering will cost $20 per month extra. This is a mistake. If I buy a 2GB package, they should be comfortable with my using that any way I see fit for personal use. After all, I am not likely going to be using my laptop simultaneously with my iPhone. This is ATT being greedy. They just want a per device charge, but a per-person charge is actually more rational and customer friendly. Tethering has enough shortcomings in battery life and convenience to be its own punishment relative to buying a separate data connection for a device.
Even with this nasty fine print, the move to tiered pricing is good for ATT customers. Sure, in a perfect world there would be enough capacity that everyone could just use as much as they want, but the reality is that wireless data is a scarce resource today for ATT, and by charging people for what they consume, they will better allocate their resources among their customers.
How does this affect photography? Cameras are used sporadically. Allowing consumers to pay for the data they use will allow cameras to get cellular data connections that don’t need to cost anything when you don’t use the camera. Today, some devices are already sold this way, like the Kindle. Of course, ATT is not selling iPhone data like that today. It’s use it or lose it, but maybe someday they will. This is what is needed to put cellular connections on every device on earth.
HD Video for All
We are pleased to announce that we have changed our pricing plans to now offer HD video to all customers, a feature previously offered only to Phanfare Pro customers.
Here is a full summary of the additional features now being offered to our Phanfare Premium customers:
- HD Video. Display your videos on the web in 720p HD. HD videos look gorgeous, especially fullscreen.
- CNAME Support. You can use your own domain name with Phanfare and display your albums at http://www.you.com.
- More Customization. Add your own custom header or footer. Change our colors to match yours.
- RAW Files. Store your RAW files along with your JPEG files at Phanfare. Our Lightroom and Aperture plugins support RAW+JPEG export. RAW files require purchase of RAW blocks, which are sold separately.
- No Phanfare Branding on Your Albums. This feature takes me back since it’s the way Phanfare was launched in 2004. 😉
- Subsites. (announced last week). You can have an unlimited number of Phanfare subsites at you.phanfare.com/subsitename, each with optional password protection and its own title and description.
We are making these changes because we want Phanfare Premium to provide everything a photographer needs to organize, archive and publish his photos and videos.
This amounts to a re-segmentation of the Phanfare product offerings. Phanfare Premium is for enthusisasts and Prosumers. Phanfare Pro is intended for working Pros. The Phanfare Pro plan allows working photographers to monetize their work through the sale of photo merchandise. We pay 85% of the markup to the photographer.
Phanfare Price Increases
We raised our subscription prices today for new customers and renewals. The new prices reflect the value we deliver to customers and will increase our margins to the point where we can continue to invest in the business.
Phanfare stores all customer photos and videos at Amazon S3 and we encourage customers to store all their media with us. We store the photos in their original quality and the video in HD quality. The average customer has 10GB of data. Amazon’s S3 pricing is well advertised.
Amazon provides the quality of storage that our customers expect. Data is kept in multiple data centers and the likelihood of their losing a single piece of data is so negligible as to not be worth discussing. The data will be there. And it’s all near-storage, not tape backup.
What does storing your photos in “original quality” mean? Well, if we got rid of photo renditions larger than what we use in our slideshow, we could remove 90% of our data costs. But if you are a customer of Phanfare, you don’t take photos to have us lose 90% of the data. Phanfare is the service that won’t have to upscale your image to show it on screens that are 600 DPI in ten years.
Unlike many services, Phanfare treats the data as if we have the only copy. We are not secondary; we are primary. Even backup services, that sell based on the fear of losing all your data, typically don’t keep the data in storage built to endure data center fires. Backup services expect you to keep a copy as well. Not true for us. Our vision is to be your cloud-based photo and video asset management system. Computers, mobile phones and tablets become cache-coherent terminals. Cameras become connected acquisition devices.
The decision to keep archival originals, which is one of our primary differentiating characteristics, drives our cost position. But Phanfare, as I blogged in 2006, was never designed to be the low cost leader. In a world of free, low-fidelity photo services with no customer support and me-too features, we try to produce a service that is better, not cheaper.
Phanfare enjoyed some margin at our old prices, but it was very thin. At enormous scale, the model was fine, but at the size we are today, and the size we will likely be in the near future, we require much healthier margins to invest in the business. If you are our customer, you want us to have that margin. Really, you do. Because the alternative is not pretty.
Today’s price increases merely represents our acceptance of the reality that we are willing to trade growth for the profitability and sustainability of the business. Our first obligation is to run Phanfare in a way that is responsible and enables our long term success. We are the steward of our customers’ memories.
I truly believe that there is a huge opportunity to build a trusted brand in photo and video management to last the next fifty years. Kodak had a strong enough brand to do it, but they squandered the opportunity. I am talking about a brand strong enough that you know that if you pay the company, your photos and videos will outlive you. We are not there today, but I want to try to get there. And the way to do it is to charge prices that give the company a healthy margin so it can control its own destiny.
Phanfare Subsites Explained
This Thursday we plan to remove the Phanfare Social Network, the ability to add friends, family and create groups within Phanfare (read just the bold parts of this post for the Executive Summary). We are doing this because the social network never really caught on, is a bit confusing, and requires viewers to maintain registration with Phanfare that offers them nothing more than the ability to see your photos and videos.
As part of this release, we are offering Phanfare Subsites to all Phanfare customers. Today, Phanfare Subsites are available only to Pro customers.
With each Phanfare account comes your own Phanfare site, located at you.phanfare.com, where you is whatever you want it to be. You can designate any subset of your albums to appear on your site. All well and good.
But what about when you want to share albums with a different interest group? For example, let’s say you take photos at your kid’s soccer games. You want to show those albums to other parents, but you don’t want to expose them to all your personal photos. Enter subsites. You can can create a subsite called you.phanfare.com/soccer. Within Phanfare, you can designate which albums are published to the soccer subsite.
An album can be in multiple subsites and/or in your main site. You can publish the soccer-related albums to your main site and the soccer subsite, and you probably would if the soccer photos are of your own kid.
Each Phanfare site or subsite can optionally be protected by a password. Hence, you can lockdown your main site but decide to keep the soccer photos open for the convenience of the parents. You might decide to also suppress google indexing of the subsite so that random people are not likely to find it.
But wait, you say, one of the things you loved about our Social Networking was the automatic notifications out to your network when you created new albums (we know, we just did a survey and of the 8% of our customers who will miss social networking, 80% say what they will miss most is automatic notifications).
We are going to solve that for the release this Thursday. We are adding back in automatic notifications for those who want them. The Phanfare contact manager enables you to create groups within it. You will be able to specify that any particular group of people be automatically notified when you publish content to particular sites or subsites. For example, you can create a contact list called “family” and have them get an update whenever you publish an album to your main site.
Because nearly all viewers come through email messages that are generated by Phanfare, either automatically or on-demand, viewer reports will mostly work as before. We click-track when viewers click through and report on who clicked and what they viewed.
So what’s the difference between using subsites versus our old social networking features?
- With social networking you had to get the person to accept the invitation, create an account and remember his password in certain scenarios.
- With social networking, you could disconnect from a single person without any disruption to the rest of your viewers.
- With social networking, to move a person from one group to another, you had to remove them, add them to the new one and wait for them to accept the new invitation to join. (And since mom does not even realize she is registered for Phanfare, good luck with that!)
- With Phanfare groups, you could create a collaborative group where everyone could contribute content (although only 5% of groups were used this way).
- With subsites, to turn off a single viewer, you need to change the subsite password. But for many situations, simply removing them from the list of people who get notifications might be enough. For example, you don’t want to notify someone you have fallen out of touch with when you add new albums but don’t care if they proactively seek to look at your photos. And since our notifications just work on clickthrough, most viewers might not notice a change of subsite or site password.
- The site hosting model in general works better for offline to online. For example, you can create a subsite called you.phanfare.com/holiday and have it be some highlights from the last year. Print that on a holiday card with a subsite password and recipients can find the online content. This is simply not possible with social networking since recipients need unique registrations.
Subsites are also useful for project work. If I want to host an album of photos showing storm damage for my insurance adjuster, I can do it using a quick subsite. I create the url: me.phanfare.com/storm, publish the storm album into it and send a link via email.
On thursday, customers who regularly use the Phanfare social network will receive a note pointing them to a wizard to migrate over to the use of subsites. For friends, family and each group they maintain, we will suggest a subsite, and automatically publish the appropriate albums into the subsite. We will also create a contact manager group and give you the opportunity to specify that recipients should receive automatic notifications when you add albums to the subsites.
For your viewers, not much will change. Instead of receiving an notice from Phanfare daily with all the new albums created by their connections, the viewer will receive one email per album published, with the From address of the email being from you. One nice benefit: the recipient can reply back to the email to reach you. The recipient won’t need to know any passwords (we encode them on the clickthrough) and the recipient will see only your albums when they click.
These changes don’t affect your ability to send isolated links to single albums that live outside your site structure, or transfer albums to facebook.
With the removal of the Phanfare social network, we complete the migration back to our roots of being an asset manager and publishing service for photo enthusiasts and pros. We have no plans to offer a free version of Phanfare, and we don’t need or want the viewers of our customers to register with us.
We hope the product is just that much easier to use and reason about for our customers and that we will have more resources to bring to bear against the problems that all of you have told us are your higher priorities for Phanfare, including but not limited to: keyword tagging, faster search and wider mobile integration.
The Panasonic GF-1 heralds the second rise of the point and shoot
Photography enthusiasts of a certain age remember that it was not long ago that the SLR camera was declared all but dead, a niche product for die hard tinkerers. The date was 1995. Film was the name of the game. Point and shoot cameras (P&S) were getting better and better. Enthusiasts were buying Yashica T4 cameras and leaving their heavy iron at home. I remember a series of articles by Philip Greenspun, founder of photo.net, talking about point and shoot cameras being more than adequate for most purposes (some of those pages have been updated).
The thinking went something like this: Most people buy P&S cameras and hence there are more R&D dollars to develop them. P&S cameras were improving at a faster rate than SLR cameras and you could see the day when the quality of the images and auto-focus systems would mostly equal that of the expensive cameras. Back in film days there was no difference between the light sensitivity of P&S cameras and SLR cameras since they both used the same film.
Cannibalization from the low-end is a common phenomenon in technology. As technology improves and prices come down, the low end, mass market product eventually satisfies the performance needs of most applications, marginalizing the high end product. I saw this painful effect first hand when I worked for Silicon Graphics. Every year the PC graphics boards satisfied the needs of more and more people and the market for graphics workstations shrunk.
Digital Photography reset the camera market. Camera prices more than doubled overnight. In 1999, entry level P&S cameras were $700. Digital SLRs that could rival film were $10,000. For all the enthusiasts moving over from digital, there were some painful choices to make. Digital had clear advantages in immediacy and the incremental cost of shooting, but most enthusiasts were priced out of the cameras that could deliver image quality equal to their $700 Canon A2E film camera.
As prices dropped and technology improved, Digital SLRs became the tool of choice for the enthusiast. Starting with the Canon D30 in May of 2000, which was priced at $2400, enthusiasts gradually started buying digital SLR cameras.
Digital SLR cameras came down in price over the years. Now once again, digital SLRs cost approximately what prosumer film SLRs cost in the 90s ($700-$900). In the last few years DSLRs were one of the fastest growing segments of the digital camera market. Only a digital SLR could offer the shot-to-shot time, auto-focus speed, and low light performance that enthusiasts demanded.
But there is no fundamental technological advantage to the SLR format where you look through the lens through a pentaprism equipped with a mirror. In fact, the whole concept of having a mechanical mirror that pops up to expose the sensor is a complicated mechanical contraption that seems almost odd in a modern digital camera. Furthermore, the SLR format has some disadvantages, including size, weight and frame rate (you have to move that mirror out of the way).
Why can’t point and shoot cameras produce images that are as good as an SLR in a smaller form factor? Well the answer is that they can. Panasonic and Olympus have led here with the introduction of the micro 4/3rd format, which is really nothing more than a line of point and shoot cameras with interchangeable lenses and big image sensors.
The Panasonic GF-1, which I own, is the first camera that makes me want to leave my 4.5 lb Canon 5D Mark II with 24-70 f/2.8L at home in some situations. Not all situations mind you. But some. the GF-1 is 1lb with its 20mm f/1.7 lens. It can take a photo in low light. It autofocuses well. Challenges remain. Auto-focus speed is not equivalent to what a DSLR can deliver. Low light performance is not equivalent to a Canon 5D Mark II. But you can see where this is going.
DSLRs are not getting better at any significant rate. They are already amazing. The gap between P&S camera performance and DSLR performance is closing. When P&S cameras deliver anything close to the performance (image quality, low light performance, auto-focus speed) of SLR cameras, the market will once again shift back to point and shoot cameras.
Why? Because consumers mostly don’t care about tinkering with settings (aperture, shutter speed). They care about image quality, auto-focus speed, and low light performance. Once point and shoot cameras close the gap, the market will shift away from the heavy, clumsy digital SLR cameras.
I believe that when we look back, Panasonic’s GF-1 will be seen in the industry as heralding the second rise of the point and shoot camera. In five years, I predict the DSLR market will actually have shrunk relative to the market for compact, 1 lb point and shoot cameras with digital viewfinders and amazing performance. These cameras will be under $400.
And after that? well, technology is merciless. Don’t count the smart phones out. It will just take a long time before they satisfy the performance needs of the mainstream.
Danger data loss give hosted services a bad name
Last week we learned that Danger, a subsidiary of Microsoft, has lost huge amounts of customer data. Danger makes the sidekick smartphone, and they offer a service to synchronize the phone (contacts, photos, etc) to hosted servers, aka, the Cloud. The critics wanted to know “why was there no backup?” And of course then there was the inevitable refrain that if you want to keep your data, you should be backing it up yourself and not relying on cloud services.
I think this is entirely wrong. Using a cloud service should free the consumer from having to do backup. Most times when you use a cloud service, backup is not even possible. How do you backup your gmail account? How about your facebook account?
The whole reason to use a hosted service is to free you of having to deal with the muck of running your own servers and doing backup. It makes sense precisely because building a reliable service is so difficult.
I am not on the inside at Danger, but I think I know how this happened. While people think that Danger lost a lot of data, the truth is, they lost very little. The type of data they hosted (contacts, text emails) is small compared to photo and video data, of which they had relatively little. My guess is that they lost under 10 terabytes of data. It might have been under a terabyte. And when you don’t host huge amounts of data, you might be tempted to just put it on RAID’ed servers and try to do nightly backups. Turns out, its very hard not to lose all your data when you use RAID.
RAID pretty much requires you to run nightly backups or rely on a proprietary replication scheme. RAID is sold as being completely reliable but anyone who has used RAID knows this is far from true. Double disk failures are more common than expected, especially when drives are from the same lot. Sometimes you lose the whole RAID chain. Replacement of disks is a manual process and sometimes people replace the wrong disk. Corruption of a RAID volume is not unheard of.
Then there is the backup window, which becomes longer each day until you finally start spending more hours backing up the data than there are hours in the day (been there, done that). And when backups are occurring, the performance of the RAID is significantly degraded. In sum, RAID does not scale. And any service that gets large enough eventually abandons RAID for some distributed solution that scales better, and coincidentally, is a lot more resistant to losing all the data at once.
Phanfare uses Amazon S3 for storage of photos and videos. Amazon is fairly vague on how it works, and we are under NDA, but the basic story is that it works much like other modern distributed file systems. It keeps multiple copies on multiple servers, geographically distributed, and has a scheme for replicating data when it programatically detects that a copy of an object has been lost.
As such, there are no backups of Phanfare. Yup, that’s right. We don’t backup the image and video data. It’s on Amazon S3 and that system uses an approach to persistence that is fundamentally different than the approach that bit Danger in the you know what.
Truth is, backups serve two purposes in am modern system. They do help assure that you don’t lose data to a system problem. And they serve as checkpoint against human error of deliberately deleting data.
The problem with S3 is, when you give it to the command to delete a file, it gets deleted, reliably. There is no going back to last night’s checkpoint. To combat this issue, we don’t really run deletes when end users delete their images. We wait a while. And we have a trash can system to make absolutely sure you want to delete data. Waiting on the deletes is really to protect against a systemic failure on our part (rogue code that deletes files).
We still use some RAID storage at Phanfare for some relational database systems holding meta data. The web service caches this data using memcache. (Another rule of large scale systems is that relational databases don’t scale either). At some point, we will scale past being able to use RAID and caching for that. Until that point, we do have to perform old school backups of the relational database to a secondary data center. And I worry a lot more about those than I do about the image and video data at Amazon S3.
The whole Danger incident sends the wrong message. Companies are much better at keeping data reliably compared to consumers. That Danger dropped the ball should not indict the whole industry. Instead, consumers should demand that companies be more transparent about their approaches to keeping data reliably.
In recognition that the ultimate risk is always that you do not know all the risks, we also offer a DVD subscription service that returns your data to you incrementally over time, automatically, so both we have and you have it.
An Engineer 