New Ideas, From Near and Far

NPR ran a story recently on why we miss creative ideas that are right in front of us. Summarizing the research, people rate ideas that they believe came from far away as more creative than ideas they believe came from close by. As someone who has frequently tried to sell new ideas to coworkers and management, this seems pretty plausible as one of the many obstacles to change.

Resisting Innovation

Almost all companies, from the CEO to HR to the customer support group, say they support innovation in all areas of the company. Who doesn’t love new ideas?

Well, the truth is almost everyone fears and resists change. In our tech-driven culture, it’s become almost politically incorrect to suggest that disruptive innovation isn’t welcome, but that’s the reality. People like to do what they have always done. When a business is involved they will point to “what has always worked” and fight to maintain the status quo for fear that any changes will destroy the business.

So the first challenge you face is that despite the stated love for new ideas, most people will try to shut them down regardless of whether they came from near or far.

Not Invented Here

New ideas you may have picked up from some outside source can also run into the commonly observed “Not invented here” syndrome. This is resistance to ideas, products, or solutions that have come from outside the company because of the belief that the people in the company can do better (and a host of other reasons).

This seems to contradict the far and near research, but in the NPR story, Vedantam suggests one theory for the different reactions to a new idea is your frame of mind when considering it. Things nearby seem more concrete, leading you to think about things like implementation details which are more likely to lead you to shoot holes in the new idea. When you’re thinking about something that came from far away, however, you’re “in a more abstract frame of mind” which allows you to think about possibilities without getting bogged down in details.

Presenting solutions that involve tools (like software packages) or techniques from outside the company may quickly lead technical people to thinking about the concrete details, making them more likely to see all of the problems. So the “Not invented here” reaction could be inspired by the same dynamics as those demonstrated in the near and far research.

Ask the Expert

So what’s the answer? Just forget your new ideas?

One approach I’ve had success with is finding outside consultants to help sell a new idea and possibly help with implementation. Even if you are an expert in the topic and you know how to implement the idea yourself, bringing in outside consultants who believe in the idea as much as you do can be effective.

Consultants can push through resistance in two ways. They can provide the “idea from far away” that can help management think about the idea abstractly and see the possibilities. If you can find consultants from out of town who have to fly in or webex to help with the pitch, even better.

On the reverse side, if your consultants are well known experts in their fields–and they should be–they can help overcome some of the resistance from co-workers worried about implementation details. Consultants have the experience of having successfully implemented the idea before and your co-workers will likely get a kick out of working with experts in the field.

So consultants can provide the far away perspective to help the big picture selling of the idea and the real-world issues of implementing it. On top of that, your manager might just be thinking about all the work you aren’t going to be able to get done if you’re working on the new idea. Consultants don’t invoke the same sort of resistance. Using consultants for part of the implementation allows you to come up with a fixed cost that ends when the engagement is over.

Implementing change and getting buy-in on new ideas is a popular topic, so much so that there a whole industry around it including popular books. This is just one idea I’ve had success with. I’m not a consultant and don’t have any vested interest in promoting them, but the way they can act as agents of change can make them useful even when you already know what you want to do—maybe especially when you already know.

If you’re interested in more on the rise of consulting, the folks at Freakonomics did a podcast on it. Good luck with your new ideas!

The Winter Olympics and 4 Years of TV Everywhere

As the Winter Olympics approach, I can’t help but think back to the Vancouver Olympics in 2010. Although I enjoy the Olympics, it’s memorable for me because I headed up a team at Synacor, Inc., at the time and we effectively launched TV Everywhere online during the Vancouver games. HBOGo was soon to follow, but the Olympics was the first real widespread rollout.

Online Olympics

During the summer of 2009, NBC started talking to cable companies and technology providers like Synacor about streaming live Olympic content online for the Vancouver Olympics. TV Everywhere had already started with some experiments, but nothing had been done across the full subscriber base of cable providers and telcos. NBC had an ambitious plan to try to make content available online to everyone who was paying their cable company to watch it on TV.

It was a bit strange that NBC, a broadcast network, would launch an initiative that required a cable subscription. But it shows how much even the broadcast networks rely on cable revenue and of course the rights to the Olympics aren’t cheap. As it turned out, access was based on being subscribed to CNBC and MSNBC, the network’s cable properties.

Managing Access

Synacor is a technology provider for many cable companies and telcos and once we had the basic plan from NBC, we needed to reach out to every customer and figure out how we were going to allow their subscribers to log in to view the Olympics. We had a base of people who already had a login because they used the email service provided via their provider’s broadband package, and we provided the email service. But what about checking what channels they had in their line-up? And then the real curveball, what about video subscribers who didn’t have a broadband package? Or even worse, what about subscribers who got broadband from a telco (DSL) and video from the cable company?

Needless to say, we had our hands full. I spent months on the phone with the tech teams at all of the providers figuring out how to get access to their subscriber data. We needed to map channel line-ups to existing accounts, have a way for new users to provision new accounts, and allow users who didn’t even know they had accounts to reset their passwords.

One of the hardest parts was creating accounts for TV-only subscribers. There were technical challenges because many of the online systems were tied to the user having a broadband internet subscription. Even more difficult was convincing people that these subscribers would even have a way to access the content. How can they watch online video without our broadband account?

Bolting Things Together

The Olympics run on fixed dates and they weren’t going to push them back a few days if we weren’t ready. It was a scramble right down to the last minute to get people set up. We ended up with 14 of our customers signing on and it was a full-time effort to get everything in place.

The authentication and authorization process was a federated identity system using SAML. Basically that means setting up a trust relationship between two parties (web sites) such that a user can get logged in with one (Synacor) to get access to content at another (NBC). Getting one of our clients set up first involved getting all of the user and channel information on a regular schedule from their back-end business system into our identity system. Then we needed to exchange identity information and metadata with NBC to register that customer’s login page with the NBC Olympics website.

It was hard enough to do this for one login, and we had to do 14. NBC set up a partner portal to help with the process and by the end, my team had the process down pat. Regardless, we were still putting things in place at the last minute as our customers rushed to complete things on their end.

After a few late nights, my team had everything in place and we waited and watched our monitoring systems as the opening ceremonies started.

Let the Games Begin

Thanks to the efforts of my team, we launched on time. We watched and worried and wondered if we had enough capacity to handle the load. We wondered what would happen if the many backend systems we had integrated started to slow down or stopped working altogether. And we looked ahead to the men’s hockey final at the end of the games and wondered what the volume would be 15 minutes before the gold medal game started.

As it turned out, we didn’t need to worry. For us, the load never used more than 10% of our capacity. And as the games rolled forward, other providers did have problems, so rather than risk having angry subscribers, NBC eventually opened up access to some of the more popular events. Regardless, we were happy to be able to say that throughout the events all of our customers had fast logins with no problems.

Still Work to be Done

The online numbers generally were low and there are a few reasons. For the U.S., the games were close to the same time zone making it easy for NBC to put events on linear TV during times when people were home. The events were limited to hockey and curling, so not all content was available. Finally, sports in general has a different usage profile than other content. People want to watch it live and the audience is much lower for replays, especially once the outcome is widely known.

From a technical perspective, I viewed it as a big success. We integrated a large number of systems and the technology worked. Users who knew their credentials were able to get in and watch video and other exclusive content.

But other lessons we learned then are still being sorted out and they are essentially the lessons around web identity in general. How can we make it easy for people to log in? Users need to know their credentials, need to be able to self-provision if the don’t have them, and retrieve passwords when they have forgotten. And as much as providers don’t like it, they need to allow users to select their own identity provider even if it’s someone else.

As the winter games are gearing up, NBC has announced they will allow 30-minute free passes to get people engaged with TV Everywhere. Unfortunately, I think the issue is that it’s still too hard for legitimate customers to get logged in. Providers need to streamline their systems, make it easier for users to learn their credentials, and allow subscribers to use other logins they know like Google or Facebook. I’ll be watching NBC’s numbers with interest to see how far TV Everywhere has come in 4 years.

Working with Software Support

I write software and over my IT career I’ve been on both sides of software support: needing support for an issue and providing support to customers for a software product. Getting support for potentially complex problems with complex software that you’ve heavily configured can be a challenge. Here are some ways to make the process easier for people on both sides of the support system.

What’s Wrong?

Most support starts off with the customer describing some issue they’re having in text and sending it off to the support organization. This might be an email alias like support@example.com or a web form in a support portal. The point is it usually starts off with text, which is a really rough way to describe a detailed problem.

Where do you start? Here’s the standard initial points you need to address first:

  1. What did you do? Give the minimum number of steps to reproduce what you’re going to mention in 3.
  2. What did you expect to happen?
  3. What happened?

This covers the basics, helps you focus the issue on exactly what’s wrong, and allows the support person to start to focus on “our software doesn’t do that” or “yeah, that’s not right”. If the software has a GUI, screenshots can provide a lot of information easily.

After the basics, you may want to add some more detail about the issue. If you’ve seen advice that you should hire developers who have excellent writing skills, writing bug reports for support is one of the reasons. Whether you’re filing a bug report for another developer on your team or sending off a support request, you need good writing skills to accurately and concisely describe a complex issue.

Fundamentals

If you want to avoid a guaranteed back-and-forth at the beginning, you should also provide as much of your environment information as possible up front. This includes the version of the software you’re running, including any updates or patches. Provide the system it’s running on, whether it’s RedHat Enterprise Linux 6 or Mac OS X 10.7.5 Lion.

Then include any other relevant diagnostic information. If the software has a configuration page, send along a copy or screenshot. If it has logs that you’re aware of, send along a relevant section gathered as you demonstrated the error. You may not see anything useful there, but even the absence of messages can be helpful for the support person.

Provide background on any history if there is one. You may have had similar issues in the past and support gave you a patch or update. Support may have provided steps to fix something similar previously, but the fix isn’t working this time. Provide support ticket numbers if you have them just in case the new support person couldn’t find them.

You don’t want to overwhelm the support person, but lastly you can include what you’ve tried so far. You may still get the standard “try this” response, but if you get a human on the first try, you might get bumped up to an engineer if you’ve already hit all of the obvious first steps (clear your cache, reboot, apply the latest system updates, etc.).

Following Directions

You’re probably working really hard to fix the issue on your own, which is why I suggest telling support everything you’ve tried. But once you send off the support request, it’s time to stop trying things randomly on your own. The support person is going to try to reproduce your issue and it can become very difficult if you become a moving target. In the best case, you’ll fix your issue but you won’t quite know why, which means it could happen again. In the worse case, you’ll cause another issue and then you’re really in a bind because you can’t easily answer 1, 2, and 3 from above because you were “just trying things.”

Assuming you have a decent support company and your boss has paid for a reasonable SLA, you should get a response in a reasonable amount of time. Unless it’s a really common issue, they are likely to send some instructions that will either give them some additional information or potentially solve the problem.

Do what they ask.

One of the more puzzling things I’ve encountered is the frequency with which support customers ask for help, then do everything but what you suggest. The support person is trying to piece together your problem from only a few clues and when they ask for something, they probably have an idea what’s wrong. They may be trying to rule out a known bug or confirm a particular setting or detail. When you do what they ask, you help them move along their train of thought. When you try a bunch of other things, they essentially need to reset and start again with how your system is set up now.

The Whole Story

You’re sitting there waiting for a response, your boss is bugging you every 15 minutes for an update, and I’m saying don’t try things. It’s understandable if you get a little antsy, especially if you get an idea after dwelling on the problem for a few hours or days. If you do make changes, keep track of exactly what you’ve done and let the support person know on the next exchange. Yeah, maybe you did something you shouldn’t have. Keeping it a secret just compounds the mistake.

Likewise, if you’ve been gathering information from other people on the issue, send along new information as you learn it. This is especially important if you track something down and find out you got incorrect information previously. Support may be holding off on sending you a fix because it doesn’t make sense given that part of the puzzle.

Closing

If you’re lucky, you’ll get a good support person and they’ll get you a fix. It might be a patch or update, a configuration change, or a reboot. Whatever the case, when the issue is resolved and you’re back up and running, reply back so the support person can close the ticket. Some support organizations are evaluated based on closing tickets. Knowing you’re all set helps them keep their ticket system cleaned up. Some systems auto-close after a period of inactivity, but the ticket history won’t reflect whether you got things fixed, especially if the last response included several things for you to try.

We all rely on software and it can be panic-inducing when it becomes uncooperative. Having good support available is one safety measure. Knowing how to work with them effectively is another.

Setting Up a Blog

Seth Godin‘s advice for people is to blog everyday. He doesn’t understand why everyone doesn’t do it. Well, I’m finally taking his advice and I think I can offer some insight into what stands in the way.

First Steps

I’ve always thought it would be a good idea to blog, but after attending a Seth Godin talk where he repeated his blogging advice, I decided to finally get going. But where to start?

I write software for a living, so of course my first thought is to set up something self-hosted because, you know, I might want to hack on the source. Being mostly a Perl guy, I started searching for something in Perl and found Movable Type is still a leading blog platform. I found the community site and the project on github. As I read more, however, I ran across the blog posts about license changes and found them all very confusing. Looking at the github project, I couldn’t find any clear software license referenced in the README. When I found the license purchase options, the minimum is a 5-user license, so it seems they are now focused on business users and not individual bloggers.

After spending a few days reading about all of this I realized I was allowing myself to get distracted from the main goal, setting up a blog. Did I really need to hack on the source? Of course not, the goal is to get up a blog post somewhere.

Hosted Blogs

The shortest path is a hosted blog service. There are a wide array of free and fee-based services. I read David Pogue occasionally, and noticed he recently started blogging on Tumblr. I thought this was interesting for someone who essentially writes for a living, then I realized this was likely because of his move to Yahoo from the New York Times (Yahoo bought Tumblr last year).

The conventional wisdom regarding free services is that if you’re not paying for the product, you are the product. As agreements go, Tumblr’s agreement appears to be one of the better ones, with subscribers retaining rights over their content. But I was still wanted to avoid being someone’s product.

Seth Godin’s blog is on Typepad and I also use their platform for our work blog. Having a service that is 100% managed, including updates and managing security is attractive. Starting at $9 per month, and $15 per month for the “Unlimited” plan, it seemed a little much just to get something going. I also find their interface a little pushy with the “related story” suggestions.

Around this time I also caught up on some podcasts and came across an episode of InBeta discussing blogs and blogging platforms. Sadly, hearing about all of the new options out there made me even more unsure what to choose to get my blog going. Kevin Purdy’s multi-step system has the advantage of being freely hosted via github’s hosted pages, but it seemed like a fair bit of work to get something posted. The last thing I need is more barriers to getting this thing going.

Then I realized this was all another distraction. The goal is to get a blog post up!

Hosting Service

I already had some domains registered with Pair Networks via PairNIC and Pair also does hosting. Maybe that was my next stop? I looked at the hosting plans, briefly circling back to installing one of the open source products myself. Worrying I was headed down another path of fiddling with an install, I was happy to find their software installation manager.

Most of the major hosting services have these sorts of service panels now to make it easy to install open source products. Since the products are open source, the licenses are usually clear. Using the installation managers also means you don’t have to worry about running into dependency problems on the hosting platform. They have sorted all of that out.

Pair supports Movable Type in their install service, meaning I could still work with a Perl-based platform and avoid getting bogged down in the install. There’s a fair bit on the web comparing Moveable Type to the other leading blogging platform, WordPress. Most say they are generally comparable feature-wise. I revisited the Movable Type licensing terms and still couldn’t figure out what the terms really are for the open source install. Fearing some strange licensing issues when my blog becomes wildly popular, I decided on WordPress. The other factor is the number of themes. WordPress seems to have a million out there, both free and paid.

Getting all of this out of the way, I was able to register a domain with PairNIC and set up a year with Pair hosting plus software installation manager for about $120 for the year, which is comparable to the TypePad rates. The installation manager worked great, allowing me to first install Movable Type, uninstall when I changed my mind, and then install Word Press. Since I had registered my domain with Pair, it was easy to launch the blog on my domain.

Success

Getting this blog actually launched was amazingly easy once I finally sorted through all the options and figured out what I actually wanted to do. It’s true that having so many options is liberating, but also paralyzing. For now I’m happy with the outcome and I’m 100% in control. Now I just need to think about the theme, the layout, and topics for blog posts, but all of the distractions around those are for another post.