This week I gave a talk at Ignite Melbourne (Slides are below), the topic of which is how to beat google at relevancy. My opinion is that with a small team of dedicated and passionate people it is possible to build a search capability that can rival Google.
Let me first say I’m not talking about web search, at the moment Google has that market pretty much sewn up and it will take a new level of technology (possible the semantic web) to better them, however I firmly believe that if you focus on a market segment or consumer niche you can build a more relevant search experience (Step 1).
Once you have defined your niche, understand why users are searching your site, what is their goal. Users don’t search because they have a spare 10 minutes over lunch, they have a problem they need to find a solution to, search is a mechanism by which they can select a supplier to solve their problem (Step 2).
Next consider the context of the searchers query. What do you know about the searcher, the intent of the search, the device they are searching from, their location, the language they use, even the time of day they are searching, all of this information can be used to deliver an individualised search as opposed to a generic search for the masses (Step 3).
Once you understand the context of your user, think about their end goal and identify what information would allow them to compare and contrast between the results you return (Step 4).
Finally build you search algorithm so that it can dynamically take the context of the user, their intent and the content that can support the compare and contrats process and deliver results based on all of this information (Step 5).
So my 5 step process to beat google is:
1) Pick your niche
2) Understand your users goal
3) Consider all of the context you know about your user query
4) Identify the information that allows a user to compare results
5) Build an algorithm that considers all of the points noted above.
Even if you don’t want to beat Google, using the process I’ve noted you can improve on the user experience that you deliver to your users and this can never be a bad thing.
This is the video of my talk at the Lucene revolution conference in San Francisco. If you don’t want to listen to my voice the slides are availble on slideshare..
Search, APIs, capability management and the Sensis journey Presented by Craig Rees, Sensis from Lucene Revolution on Vimeo.
I’ve always thought that managing an internal capability or technology platform requires the same processes, artifacts and resources as you use in managing an external technology product such as Websphere or Vignette.
6 months ago, a couple of colleagues (Craig Lonsdale and Pete Crawford) and I were discussing our plan for improving search within our company, we wanted to document our progress as well as define where we were on our journery and where our target for the future was. I had used the CMMI framework before to quantify how well organisations delivered projects and it felt perfect to try and use the same framework in defining on well an organisation managed their search capabilities.
The Search Capability Maturity Model that we came up with looks to quantify the stages an organization goes throught as they become more mature in managing search as a true capability as opposed to just installing it and hoping it works.
Search Capability Maturity Model
Organisations move through the different levels as they grow and become more adept and knowledgable, fundamental to the model is that it’s very difficult to jump levels, I know this from experience, especially when you look at the upper levels of the model such as utilizing external collaborators.
SCMM detail
I have worked in a number of organisations and they rarely get past the adhoc level unless you are talking about the Google’s of the world and because of this they don’t get to see the huge benefits a good search capability can deliver (to be honest this model can be taken beyond search and can be used to track the management maturity of any type of capability).
Search is a capability that changes over time, search patterns change as user experience changes, language moves forward or content is updated all of this means you need to dedicate resources to managing search.
Start off easy, set up a reporting program in order to quantify how well your search is meeting user expectations. Simple things like zero results can point you in the direction of big problems. Once you have identified the obvious issues you can start to use path analysis to pick up commons search types. Focus on these and you will see your user satisfaction soar. Then in the words of agile development, lather, rinse and repeat.
There is a concept in sales call benefits based selling, this is where you sell your product or service based on the benefit it will give your customer not on the features of the product being sold. For example if I was selling you home cleaning services I would talk about the fact you could get 2 hours back a week to go to the cinema not on services I could provide such as dusting or cleaning the oven.
This same approach can be used in a search context either using a simple keyword interface or with content filters / navigators or wizards.
The key advantage of using a benefits based searching technique is that you can reduce or if your lucky remove the complexity of the search scenario. As the searcher you no longer need to be an expert in the topic you are searching, you only need to consider the reason behind your search.
For example lets look at the scenario of a consumer searching for a new camera, you could allow them to search on size of memory card, number of megapixels, size of battery or even length of lens. All of these attributes would require me to be knowledgeable about photography. How about you switch this around and allow the consumer to think about how they will use the camera. Do they want to take pictures for 3 months without downloading them, do they want to print pictures off the camera and create large posters, or do they want to take pictures of distant objects. All of these questions can be used to identify the attributes that are important to the consumer, more importantly these are all questions that can be answered without having a deep knowledge of the topic being searched. This is the advantage of benefits based searching.
Retail is a great example of how benefits based searching can deliver you consumer advantage, I think local search is the next sector in which this paradigm will take off. When you consider the disparte topics that local search sites cover, you can’t expect consumers to be experts in all of the these fields. Educating consumers with content is one mechanism to improve the search experience, benefits based searching is another. Just make sure you have the ability to link the benefits users are searching on to the attributes you have stored in your database.
When it comes to your site, I’m sure you spend a lot of time working on your UI to make it look good and ensuring functionally works but how much time do you spend on the content you publish, how do you enusre it is fit for purpose, as you roll out new features do you think about how those features will look / work when you don’t have content depth to support them.
For information rich sites the content you publish is just as important as the features you have on your site, in fact at times it doesn’t matter how great your interaction model is, if the content isn’t there you are off to a losing start.
In my opinion you have to understand what the needs of your users are and how your content will support those needs in ordre to have a chance of success. To do this you need to define a framework by which you can qualitatively and quantitatively measure whether you are meeting those users’ needs.
Path analysis, satisfaction surveys, competitor analysis can all be used to define the needs of your consumers (or your perspective consumers if you haven’t launched yet), once you have identified the primary needs you can then identify the types of content you need to publish to support those needs. Once you have defined the needs of your users you can quantify how well you currently meet those needs, where the gaps are and what you need to fill those gaps. You now have a framework by which you can monitor and track your improvements.
Now its all about prioritizing your content roll out plan and away you go.
Search hasn’t really changed since the early years of the internet, you type in some words into a box press the search button and away you go. If you are on a Yellow Pages style site then you might have a two input boxes, one for what you are searching for and one for where you are searching for it. This means a 45 year old man searching at home on a laptop will get exactly the same results as a 17 year old school child out in the city searching on his mobile phone. Both of these individuals will have completely different needs and will find different results relevant to them.
It’s obvious that we should be giving different users different results based on the context of their search. Context can include the device being used to search with, the time of the day, month of the year, the location of the user, past search history and personal information such as a users social graph. All of these different contexts can deliver wildly different results and increase the quality or results returned.
Even context can change depending on the search being executed. For example take location, if I am looking for a pizza takeaway close by on a Friday night, location is very important to me, the difference between 5 minutes away and 25 minutes away is huge. However if I am looking for the nearest DIY shop to drive too because I want timber on a Sunday morning because I am building a deck the difference between 30 minutes and 45 minutes may not bother me. This information can help the search engine determine what is the most relevant result.
Over the next few years we will see more complex search engines being developed, they will gather as much information that they can about the searcher and deliver tailored results to a query based on the context of both the query and the searcher.
What with Google place search, Groupon types sites, Paypal bricks and mortar payments and Facebook launching deals and venue search, the local search ecosystem is coellecing together, but search is just a means to an end.
A consumer has a need that can be serviced by a local business, they don’t care how they find the business or if it was recommed by a close friend or if 99 other people use that service they get 10% off the price, what they really care about is fulfilling their need whether it be to take their partner out for an aniversairy meal or gettng an extension built to house their growing family. The ecosytem is not about local search but about local commerce and this is what Yellow Pages companies were all about, to bring buyers and sellers together.
The local commerce ecosystem is made up of a number of key components:
- Search – Google places
- Coupons – Groupon
- Payments – Paypal
- Social graph – Facebook
- Check ins – Foursquare
- Reviews – Yelp
It seems that they is an opportunity to deliver an end to end capability across the local commerce ecosystem, but not as a stand alone platform. Web 2.0 / 3.0 has show us that the web is no longer about monolithic stand alone sites, its about integration and interpolation across different destinations, platforms and devices, its about the glue that brings it all together and maybe this is where Yellow Pages has an opportunity, it can be the glue that simplifies how a consumer interacts with local businesses, a friendly trusted guide across this new environment.
More thoughts to follow on this topic.
This one hit a nerve because it’s title is a topic I’ve been considering for a while. What Got you here won’t get you there by Marshall Goldsmith talks about the pitfalls you can fall into after a bit of success in your career. You think all of those traits and habits that got you where you are today are the same ones that can get you to the top of the ladder, well you were probably wrong. Great read and very thought provoking. Thank you to Becs for pointing it out to me.
Google has just launched their new search UI after months of A/B testing, for me the biggest change from a functionality perspective is the prominence given to the search refinements options (now occupying the new left hand column). You can refine your results by a particular time period, the type of media you are looking for or even particular forms of information such as reviews.
Google serp
Search refinement or feature filtering as it’s also called is not a new concept, online retailers have been using this feature for a while, think about buying a TV, you might filter on the type of panel, the size of the screen, the types of input or the resolution. You might have also seen this type of functionality being utilized on recruitment, car sales and holiday web sites as seen below.
Search refinement is a mechanism that allow users to create complex boolean type search queries using a graphical interface. It allows a user to filter down a large set of data based on a number of different attributes, great for when a user has an idea about what they are looking for but needs a guiding hand on getting to their correct result. It a bit like a crossing a browse feature withe a search feature and getting the best of both worlds.
So far, local search players have not jumped on this band wagon, the major players still use a single search box a la Google old school, is this because they don’t feel that it has value or perhaps that they don’t have the type of structured data required to drive a refinement style capability or the search platform to deliver this capability in a cost effective manner? Without structured data such as product catalogues or defined service style options it can be difficult to deliver a successful refine by interface, especially when you want to guide a user to a successful result and not to lead them down a blind alley into a no results page.
As we have seen in the past Google drives users expectations, as users become comfortable using this type of feature when doing a general web search they will start to expect it when they are doing any other type of onsite search, so my view it that it is more of a when rather than if this type of capability will be delivered, so start to get your data structured, it can be a slow and painful process but your user base will thank you for you hard work.
How many great leaders have you worked for? I don’t know about you but my list is quite short.
First let me be clear, I’m not talking about good managers, because to me that’s something different. Leadership is the person I am compelled to follow, someone who can stand in front of me and with their words and their passion can raise my emotions and make me believe again when I am at my lowest ebb. They are the person you look to when a key decision should be made or a direction set.
A few months ago I read an article that proposed management can be learnt but leadership is something you are born with, I don’t want to believe this statement but the lack of true leaders in the organisations I have worked makes me reconsider my position.
One of the reasons I feel organisations fail is because of a lack of leadership. I have worked at a number of places where ideas are generated hand over fist but decisions aren’t made, directions are set and delivery doesn’t happen and this is where leadership comes in. You can’t be a successful delivery organisation without true leadership, its like an army without a general, they may be great one on one but when it comes to winning the war you need someone setting direction from the front, remember the old saying that a camel is a horse defined by a committee, well when was the last time a committee ever delivered anything of a value? If they do deliver it tends to be because a single individual stood up and took on the leadership of the group.
So next time the proverbial hits the fan, or your project hits a brick wall or your share price drops 40%, or your company misses its sales targets, ask yourself who are people turning to, who are they looking at for an answer, perhaps it is you or maybe its an opportunity to step forward, perhaps leadership is just about a person being willing to take a risk and make a call, if this is the case then perhaps anyone can be a leader.