An Engineer

An Instance of Perspective

Archive for October 2009

I just cancelled my Zagat subscription. Those guys got Yelped.

with 9 comments

I have been a subscriber to Zagat.com for about 10 years. It costs $24.95/year. They provide online access to the information that is in the books. As New Yorkers know, where the guide got its start, The Zagat survey was for a long time the goto book for figuring out where to eat in NYC.

They put the ratings together through an annual survey that diners completed.

They charged for the book and when they went online, they charged for online access to the same data. Why? Because they did not want to cannibalize the sales of the book.

Well, they don’t need to worry about that anymore because I am sure book sales are close to zero. And as for online, Yelp does a better job with their free ad-supported review site.

Online reviews is a natural online business. There are huge network effects. You want to review a restaurant where people are reading and you want to read reviews where there are lots of reviewers and reviews.

Yelp also has lower costs than Zagats. They don’t run a customer service team to take calls from people cancelling and they dont need to run the annual survey. The survey is ongoing and automated.

Yelp has more data too.

In just about every way, Yelp is better than Zagats now. And its free to use. So even though Zagat.com is cheap, even one penny is too much because the free product is actually better.

How did this happen to them? They were clearly the leader in 1998 when they started moving online. But their fatal mistake was an unwillingness to cannabalize the books sales when they went online. And by doing so, they left an opportunity for someone to do it to them.

I think a lot about not being Yelped. Phanfare has few network effects. It’s really high quality web hosting for your photos and videos. We are archival, holding the original bits, something I know is outside the cost model of facebook. And we are just about the photos and videos. Hence, I think a high end to our market will continue to exist: people willing to pay for a better product with better ingredients that is purpose built for the problem.

But at the same time, the mass market will go to the free ad-supported solution. For us, that is not a problem. But for Zagat it really is, because in restaurant reviews, its winner take all.

Written by erlichson

October 29, 2009 at 10:34 am

Posted in General

Why there should be no limit on the H1B visas allowed

with 5 comments

Only last year, all the H1B visa slots available to hire foreigners to work in the US were snapped up in a single day. We hired someone through the program a few years back when tech hiring was tight. It seemed ridiculous that more slots were not made available. The fear in Washington is that tech companies like us were hiring foreigners over US citizens. This could not be further from the truth.

We went looking for someone abroad because we could not find someone qualified here. We prefer to hire US citizens. There are no language or cultural barriers to overcome and it does not require an immigration lawyer.

Fast forward to today and we find out that the H1B program is now undersubscribed, according to the WSJ. Interesting, now that unemployment in the US is above 9% and there are good engineers looking for jobs, companies are once again hiring US citizens and passing over the non US citizens.

To me, this is just more evidence that there should be no limit on the number of H1B visas we allow. The cost to hire an H1B is lots of paperwork and at least $5000 in legal bills. That tax alone guarantees preference for US workers. Beyond that, limits only make US tech companies less competitive in boom times.

Truth be told, I am pretty pro free market. I think the market allocates resources somewhat efficiently and when we tinker, we always create some secondary effect that may or may not be desirable. Rather than prop up dying industries with stimulus money or try to pick winners in new industries, I would have sent the millions of unemployed workers back to school to retrain them in industries that are expanding. Ultimately, our prosperity will be related to our individual productivity multiplied by the number of people working.

Disclaimer: this is my blog and these are my opinions. They don’t necessarily represent the views of every Phanfare employee. I have been scolded by readers for talking politics on this blog before, but I have a right to my opinion just like everyone else. That is what makes this America.

Written by erlichson

October 29, 2009 at 12:37 am

Posted in General

Tagged with

The Panasonic GF-1 heralds the second rise of the point and shoot

with 12 comments

Photography enthusiasts of a certain age remember that it was not long ago that the SLR camera was declared all but dead, a niche product for die hard tinkerers. The date was 1995. Film was the name of the game. Point and shoot cameras (P&S) were getting better and better. Enthusiasts were buying Yashica T4 cameras and leaving their heavy iron at home. I remember a series of articles by Philip Greenspun, founder of photo.net, talking about point and shoot cameras being more than adequate for most purposes (some of those pages have been updated).

The thinking went something like this: Most people buy P&S cameras and hence there are more R&D dollars to develop them. P&S cameras were improving at a faster rate than SLR cameras and you could see the day when the quality of the images and auto-focus systems would mostly equal that of the expensive cameras. Back in film days there was no difference between the light sensitivity of P&S cameras and SLR cameras since they both used the same film.

Cannibalization from the low-end is a common phenomenon in technology. As technology improves and prices come down, the low end, mass market product eventually satisfies the performance needs of most applications, marginalizing the high end product. I saw this painful effect first hand when I worked for Silicon Graphics. Every year the PC graphics boards satisfied the needs of more and more people and the market for graphics workstations shrunk.

Digital Photography reset the camera market. Camera prices more than doubled overnight. In 1999, entry level P&S cameras were $700. Digital SLRs that could rival film were $10,000. For all the enthusiasts moving over from digital, there were some painful choices to make. Digital had clear advantages in immediacy and the incremental cost of shooting, but most enthusiasts were priced out of the cameras that could deliver image quality equal to their $700 Canon A2E film camera.

As prices dropped and technology improved, Digital SLRs became the tool of choice for the enthusiast. Starting with the Canon D30 in May of 2000, which was priced at $2400, enthusiasts gradually started buying digital SLR cameras.

Digital SLR cameras came down in price over the years. Now once again, digital SLRs cost approximately what prosumer film SLRs cost in the 90s ($700-$900). In the last few years DSLRs were one of the fastest growing segments of the digital camera market. Only a digital SLR could offer the shot-to-shot time, auto-focus speed, and low light performance that enthusiasts demanded.

But there is no fundamental technological advantage to the SLR format where you look through the lens through a pentaprism equipped with a mirror. In fact, the whole concept of having a mechanical mirror that pops up to expose the sensor is a complicated mechanical contraption that seems almost odd in a modern digital camera. Furthermore, the SLR format has some disadvantages, including size, weight and frame rate (you have to move that mirror out of the way).

Why can’t point and shoot cameras produce images that are as good as an SLR in a smaller form factor? Well the answer is that they can. Panasonic and Olympus have led here with the introduction of the micro 4/3rd format, which is really nothing more than a line of point and shoot cameras with interchangeable lenses and big image sensors.

The Panasonic GF-1, which I own, is the first camera that makes me want to leave my 4.5 lb Canon 5D Mark II with 24-70 f/2.8L at home in some situations. Not all situations mind you. But some. the GF-1 is 1lb with its 20mm f/1.7 lens. It can take a photo in low light. It autofocuses well. Challenges remain. Auto-focus speed is not equivalent to what a DSLR can deliver. Low light performance is not equivalent to a Canon 5D Mark II. But you can see where this is going.

DSLRs are not getting better at any significant rate. They are already amazing. The gap between P&S camera performance and DSLR performance is closing. When P&S cameras deliver anything close to the performance (image quality, low light performance, auto-focus speed) of SLR cameras, the market will once again shift back to point and shoot cameras.

Why? Because consumers mostly don’t care about tinkering with settings (aperture, shutter speed). They care about image quality, auto-focus speed, and low light performance. Once point and shoot cameras close the gap, the market will shift away from the heavy, clumsy digital SLR cameras.

I believe that when we look back, Panasonic’s GF-1 will be seen in the industry as heralding the second rise of the point and shoot camera. In five years, I predict the DSLR market will actually have shrunk relative to the market for compact, 1 lb point and shoot cameras with digital viewfinders and amazing performance. These cameras will be under $400.

And after that? well, technology is merciless. Don’t count the smart phones out. It will just take a long time before they satisfy the performance needs of the mainstream.

Written by erlichson

October 22, 2009 at 10:52 pm

Twitter/Phanfare integration is now live

with one comment

You can now automatically tweet when you publish an album at Phanfare. You just need to associate your Twitter account with your Phanfare account and then choose which Phanfare sites and subsites (pro feature) are covered.

When we tweet an album on your behalf, we tweet the album name and a link through a Phanfare redirector (phanfare.com/t/shortname) so the viewer knows when clicking on a link that it really comes from Phanfare.

The tweeted links don’t include any authentication so if the album is published to a site that is password protected, then the viewer will need to know the password.

Written by erlichson

October 16, 2009 at 8:56 am

Posted in General

Tagged with ,

Danger data loss give hosted services a bad name

with 9 comments

Last week we learned that Danger, a subsidiary of Microsoft, has lost huge amounts of customer data. Danger makes the sidekick smartphone, and they offer a service to synchronize the phone (contacts, photos, etc) to hosted servers, aka, the Cloud. The critics wanted to know “why was there no backup?” And of course then there was the inevitable refrain that if you want to keep your data, you should be backing it up yourself and not relying on cloud services.

I think this is entirely wrong. Using a cloud service should free the consumer from having to do backup. Most times when you use a cloud service, backup is not even possible. How do you backup your gmail account? How about your facebook account?

The whole reason to use a hosted service is to free you of having to deal with the muck of running your own servers and doing backup. It makes sense precisely because building a reliable service is so difficult.

I am not on the inside at Danger, but I think I know how this happened. While people think that Danger lost a lot of data, the truth is, they lost very little. The type of data they hosted (contacts, text emails) is small compared to photo and video data, of which they had relatively little. My guess is that they lost under 10 terabytes of data. It might have been under a terabyte. And when you don’t host huge amounts of data, you might be tempted to just put it on RAID’ed servers and try to do nightly backups. Turns out, its very hard not to lose all your data when you use RAID.

RAID pretty much requires you to run nightly backups or rely on a proprietary replication scheme. RAID is sold as being completely reliable but anyone who has used RAID knows this is far from true. Double disk failures are more common than expected, especially when drives are from the same lot. Sometimes you lose the whole RAID chain. Replacement of disks is a manual process and sometimes people replace the wrong disk. Corruption of a RAID volume is not unheard of.

Then there is the backup window, which becomes longer each day until you finally start spending more hours backing up the data than there are hours in the day (been there, done that). And when backups are occurring, the performance of the RAID is significantly degraded. In sum, RAID does not scale. And any service that gets large enough eventually abandons RAID for some distributed solution that scales better, and coincidentally, is a lot more resistant to losing all the data at once.

Phanfare uses Amazon S3 for storage of photos and videos. Amazon is fairly vague on how it works, and we are under NDA, but the basic story is that it works much like other modern distributed file systems. It keeps multiple copies on multiple servers, geographically distributed, and has a scheme for replicating data when it programatically detects that a copy of an object has been lost.

As such, there are no backups of Phanfare. Yup, that’s right. We don’t backup the image and video data. It’s on Amazon S3 and that system uses an approach to persistence that is fundamentally different than the approach that bit Danger in the you know what.

Truth is, backups serve two purposes in am modern system. They do help assure that you don’t lose data to a system problem. And they serve as checkpoint against human error of deliberately deleting data.

The problem with S3 is, when you give it to the command to delete a file, it gets deleted, reliably. There is no going back to last night’s checkpoint. To combat this issue, we don’t really run deletes when end users delete their images. We wait a while. And we have a trash can system to make absolutely sure you want to delete data. Waiting on the deletes is really to protect against a systemic failure on our part (rogue code that deletes files).

We still use some RAID storage at Phanfare for some relational database systems holding meta data. The web service caches this data using memcache. (Another rule of large scale systems is that relational databases don’t scale either). At some point, we will scale past being able to use RAID and caching for that. Until that point, we do have to perform old school backups of the relational database to a secondary data center. And I worry a lot more about those than I do about the image and video data at Amazon S3.

The whole Danger incident sends the wrong message. Companies are much better at keeping data reliably compared to consumers. That Danger dropped the ball should not indict the whole industry. Instead, consumers should demand that companies be more transparent about their approaches to keeping data reliably.

In recognition that the ultimate risk is always that you do not know all the risks, we also offer a DVD subscription service that returns your data to you incrementally over time, automatically, so both we have and you have it.

Written by erlichson

October 11, 2009 at 9:52 am

Print photos at home from your iPhone with Phanfare Photon

with 14 comments

We are announcing today that we have updated Phanfare Photon for the iPhone to enable home printing to HP printers. The updated version of the iPhone app is available in the app store now.

We worked with HP to get this all working. We are the first app outside of HP’s iPrint Photo that will allow you to print directly from the iPhone via Wi-Fi.

For Phanfare customers, Phanfare Photon wirelessly syncs your entire photo and video collection to your iPhone, enabling you to print any photo.

Phanfare Photon also allows you to order prints by mail – but that has been true for a while.

Phanfare Photon for the iPhone is our sandbox for showing how we believe connected digital cameras should work in the future. Every photo and video you take automatically floats up the cloud and every photo and video you have ever taken is available for viewing on the camera.

Truth is, most of our customers have digital SLRs, so they are more likely to view their photos and videos on their iPhone than take new ones, but we find our customers do take a small percent of their photos using their iPhone. And it is nice to be able to get a print in pinch.

This release talks about home printing, but of course, if you have a network connected HP printer at work, it works there too.

Written by erlichson

October 5, 2009 at 10:56 am

Posted in Apple

Tagged with , , ,