Imagine being lost on a deserted island with no hope of being discovered with only a volleyball named Wilson to keep you company.  There’s a reason pirates used marooning as a form of torture.  It’s a miserable existence (if you can call it that) that usually doesn’t end so well.  But yet that’s what becomes of most corporate knowledge.  It’s left on various file servers across the enterprise with little hope of discovery or rescue (aka. reuse).

In my last post entitled Strategic Reuse Process, we looked at an overall framework for analyzing how information flows through an organization and the hurdles encountered on its way to reuse.  But how does an artifact go from Publication to Discovery (see here for definition)?  In this post I want to dig a little deeper and discuss the first hurdle on our way to reuse, Findability.

find-a-bil-ity n
a.  The quality of begin locatable or navigable
b.  The degree to which a particular object is easy to discover or locate.
c.  The degree to which a system or environment supports navigation and retrieval

Peter Morville from Ambient Findability

All of the definitions above work equally well for what I am going to discuss in this post, mainly how can we describe the factors that influence the findability of information assets within the context of an enterprise.

Since we already have a working definition of findability let’s turn our attention to building a classification system to help us sort out some of the things that impact findability.  I believe that findability has two dimensions (at least) that effect our ability to locate information.  First is the degree of sharing that an object experiences and secondly is the external data associated with it over time.  The degree of sharing is quite self-explanatory so let’s look next at what we mean by external data or what I’m calling applied semantics.

Applied Semantics

Applied Semantics deals with the relationship between various signs and symbols and what can be inferred from them related to the actual artifact they represent (whew!).  Now that that nasty bit is out of the way what does it mean.  In this case I believe that the quality and quantity of external meta data that describes the original publication produces a better chance of discovery which leads to a possibility for reuse.

There are many many bits of external data that can point to an artifact and I have only just scratched the surface by defining the following five classifications:

Unclassified

This represents information that has essentially little or no external classifications applied to it.  Random documents strewn across a file system with only the author’s memory or luck as the only means for discovery are a good example.  This represents the vast majority of knowledge in the enterprise.

Cataloged

Here there has been at least some attempt made to establish external data related to an artifact.  It could be as simple as a document filing system or as elaborate as a fixed hierarchy of topics in which documents can exist.  The internals of the publication may not be readily apparent but you have some idea about what it contains from the category in which it was cataloged.

Indexed

This is where search engines come in.  They can pick apart a document and index its internals and provide an external reference to it based on the content of the document.  In theory this should work great however in practice this often fails to turn up anything of significant benefit without other external data to use in conjunction with the artifact’s content.  For instance Google relies on the number of links that point to a specific page as well as the page’s content for its PageRank algorithm.

User Applied Metadata

These are user generated data points that impart meaning on an existing publication.  They can represent tags, ISDN numbers, author name, comments, etc.  Each of these provides invaluable information that helps consumers find and use content from providers.

Solution Patterns

This represents a usage context for a particular artifact.  Some patterns of usage will be discernible from the User Applied Metadata but Solution Patterns formalize this into a language that describes when and how artifacts can be applied and under what conditions they succeed and fail in usage.

Findability Categories

Now that we have a vocabulary that helps us understand the nuances of findability classification let’s apply them to a matrix so that we can categorize artifact findability.

Missing Link

This is where most information and knowledge lives within an enterprise.  Its focus is on the individual and small teams where communication and knowledge exchange is easiest.  Examples that typify this category are personal file repositories, file shares, email inboxes, and in our developer focused example Subversion repos.  Vast amounts of corporate knowledge are stored herein but due to the lack of associated external attributes and sharing this is often as far as this knowledge ever gets.  Findability in this category is very low given that the information’s very existence is only known to a few individuals.

Paradise Lost

I gave this category the name Paradise Lost because that seems to be to be just what it is… a paradise of knowledge lost because the sharing attribute is so low.  This is where those ultra productive teams and individuals live who have developed systems to find and share knowledge and experiences but only with small groups who happen to have knowledge of the knowledge base’s existence.  Having worked in large companies for many years I can attest to the fact that some groups do an outstanding job of building knowledge bases that are tremendous assets and include features like tagging systems and user defined meta-data that help the lucky few drill into a wealth of information quickly.  However the usefulness of this is limited because few people know about it or the process for using it.  Such a shame.

Star Search

Besides the Missing Link this may be the most common category of Findability in the enterprise.  This category represents all those well intentioned projects that seek to develop a corporate knowledge base by implementing a technology solution (aka. search).  Search may seem like a great idea but at its core search is a technology solution that lacks context for relevance.  Even Google realized that when they created the PageRank algorithm that combined basic search functionality with the popularity of the artifact for added relevance.

The Matrix

The Matrix (sorry for the name but I couldn’t come up with anything better at the moment) is the best of both worlds.  It has a high degree of sharing combined with user generated meta-data and topped off with usage semantics that help organizations unlock the hidden knowledge within their corridors.  This is where communities add tremendous value in helping to detect and apply metadata to publication artifacts.

What About Communities?

If there is one thing we haven’t really discussed in this post it’s the topic of communities and their relevance to findability. Communities exist so that its members can pursue a shared goal or interest. In this context communities are invaluable in their ability to find and disseminate relevant knowledge to others in the community. Active communities can and do enhance the findability of topical artifacts, and given the right tools, and provide invaluable metadata to help establish the relevancy of corporate knowledge to specific situations. In a word they are “priceless”.

So Where Are You?

I’m curious to get your feedback on whether or not you think this framework is on or off track in helping us understand findability within an enterprise. This is one of those areas in technology discussions where opinions are like … (oops better not go there). Everyone seems to have some theory on information management and how knowledge gets created and disseminated but I’d like to hear your practical examples of how you increased the Findability of assets in your company.

Community managers have a tough job. They deal with lots of different stakeholders trying to find that elusive “middle ground”. They incessantly cheer on community activities and push adoption of collaboration best practices; but when it comes to validating their position through tangible and quantifiable metrics it can sometimes seem daunting. Is the best measure user participation? How about community size? Each of these seem like great things, and they are, but typically organizations don’t have a lot of tolerance for soft measures that don’t directly impact the “bottom-line”.

Recently I have been working to identify ways in which organizational performance gains can be tied to community activities. Since my current position involves helping large organizations increase performance from their development teams, I started first by looking at something that may seem far removed from community, knowledge reuse.

Why reuse?  The reason I chose to build my case around reuse is that the potential for productivity gains is huge. Let’s do some simple math to illustrate the point. Let’s say a development organization has 1000 developers employed. The industry average indicates that a single developer will write somewhere in the neighborhood of 3200 lines of code per year. That’s 3.2 million lines of code! At an industry cost per line of code of $27 (taken from Applied Software Measurement by Caper Jones) that’s approximately $86 million in software development costs per year!! Wow! If we can replace just a fraction of that development effort with reusable components and establish a process for interweaving reusable assets into all software projects we can save an organization millions of dollars per year.

In this post let’s first look at the process of reuse and in later posts I’ll follow up with some specific suggestions for creating an effective reuse program.

Reuse Process

Reuse has five specific phases with hurdles between each phase that impedes progress to the next stage. Overcoming these obstacles will be key to the long-term success of your reuse efforts.

Publication

During this phase knowledge is converted into something that is explicit and consumable by others. This phase represents the transfer of one person’s understanding, education, and wisdom into a tangible good. The publication format can be nearly anything – a blog, wiki, software artifact, anything that is consumable by another person.

Publication events are of very low value unless they can clear the first hurdle for reuse, Findability. Findability describes how readily available information is to other users in an enterprise. Enterprise strategies to increase findability include document repositories, index and search, push notifications, and categorization and taxonomy. Increasing the findability of assets requires a solid understanding of knowledge asset usage and user behavior.

Discovery

The Discovery phase begins when another user in the organization tries to find an answer to a problem relevant to them. During this phase the output of the publication event is found and analyzed for relevance against a new problem. This can be done via search, browsing, or by someone sharing a document via email. This is a particularly important phase because without discovery the publication event will go unused and is of very low value to the overall organization.

Understanding

During this phase information has been found but may not be fully understood by the consumer. This phase is a critical component of the Reuse Process because it promotes and encourages questioning and full understanding of the material.

Collaboration tools are essential during this phase of the process. The requisite for any collaboration tool is that a question and answer can take place to facilitate understanding between parties. To better disseminate information and to make it more accessible to others in the future, public exchanges should be used when possible. Communication channels like phone, IM, and email are great mechanisms that enable questioning and understanding but also limit the consumption of this “give and take” process to a very few parties. This sometimes is sought after behavior but in many cases the Reuse Process would be more effective if the discussions where held in public forums that are fully searchable and have meta-data such as comments and reviews that can be associated with the artifact.

Extension

The next phase of the Reuse Process is the Extension phase. In this phase we are applying the knowledge, wisdom, and education that was distilled in the original artifact into scenarios that we own and are a part of. In this way knowledge has been transferred to us from someone else and used in, perhaps, totally new ways. The context of the original information may only tangentially apply to these new scenarios, if at all. The idea being that because of someone’s published work we have been able to grow our personal knowledge base and now have the ability to extend a concept into a new area.

A great example of the extension of knowledge into new areas was the seminal work by Christopher Alexander “A Pattern Language: Towns, Buildings, Construction”. In this book Alexander makes the case for developing construction practices based on the fundamental notion that harmonious patterns have existed in architecture for centuries and that these patterns should be reused if they add value and harmony. This work was later applied to computer science by several researchers, most notably Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides and a new field in computer science was formed with the same basic tenets applied to computer science as Alexander had originally written about. This example clearly demonstrates the Extension principal (although on the extreme perhaps) that allows knowledge designed for one audience and context to be discovered, discussed, and transformed into another entirely different area of study.

Integration

The last phase is the Integration or Reintegration phase where changes that impact the original artifact are integrated back to improve the original. Integrations can take on several different forms from comments and ratings on a document to bug fixes or functionality enhancements for a software artifact.

The key to integration is to understand the need and desire of others to contribute and to provide the support and infrastructure that allows community and organizational knowledge to flow back into the original artifact.

Conclusion

As development teams organize more and more around driving value to the end customer via Agile methodologies, I see reuse as an area that could be overlooked easily without a unifying process. Project teams become focused on delivering their user stories and forget the bigger picture of component reuse because a) it was not budgeted for the sprint or b) no teams are working horizontally across the various project teams to discover reuse artifact candidates. But this post has gone on for long enough. I’ll post again soon and ferret out some of these issues.

• “too many interruptions”
• “wasting time on nonproductive tasks like email”
• “no ability to focus on key tasks”
• “excessive multitasking”

This is clearly a major problem and is probably getting worse given the increasing amount of information that keeps pouring in.  The key to solving this would appear to be to focus our attention on only the most important things.   But how is this even possible?

Attention is the most valuable commodity you control (at least from my perspective) and since I’m not one of those crazy-smart teenagers that can do 10 things simultaneously with full attention to each (my daughter apparently is though:), I need to manage my attention carefully.  The way I see it is what deserves my attention can be defined by a simple formula, something like Relevance To Me + My Trust In Source = My Attention (R+T=A).  The problem is that both Relevance and Trust are hard problems to solve.

Relevance is how pertinent, connected, or applicable something is to a given subject matter, in this case “me”.  There is so much available information today that relying on traditional methods of information discovery (search engines, RSS, etc) is just not enough anymore.  Many estimates put the number of knowledge workers in the US at between 28 and 45 percent and knowledge workers by definition require information to be effective.  Finding the right information to help knowledge workers excel is what Google’s about, right?  Well not really.  Search engines are great for finding the most popular items related to a subject but nowhere in the R+T=A equation does it say anything about popularity, so that’s not going to work.  Relevance is not a popularity contest it is very personal to you and should be handled dynamically based on many factors like whether you are at work, on a business trip, vacation, at home, working on an email, etc.  The people and places most relevant to you will be based on what you are currently doing.  Much like a spam filter learns over time what is good and bad so too should your relevancy filter learn what is important or not.  Is that Tweet relevant to the report you are working on or is it a link to a photo album of someone’s vacation?  If you’re having dinner in a restaurant should your Blackberry notify you of an incoming email?  Technology today allows many of these situations to be probabilistically identified it’s just a matter of breaking down some barriers and then SMOP (small matter of programming).

So that leaves us with Trust.  What makes a source trustworthy?  I think trustworthiness implies an existing relationship of some sort with explicit relationships having the most profound effect.  This won’t always work but may help initially filter the lion’s share of incoming data.  It’s interesting to think that a person’s social graph could be applied as a trust mechanism.  How well you trust a source depends to some extent on how far away from your node they are.  Hmmmm.  It would also be interesting to have contextual information associated with your social graph.  Given the dynamic nature of relevance, if social graph information also contained contextual information about the commonality shared by two nodes this could be very useful.  For instance if my social graph contained information about “how” I was linked to someone via work, local community, interest groups, etc, this could perhaps be used to help filter results given my current situation (ie. on vacation).  The relevance of certain connections could be elevated given my situation thus, changing the importance of the information.

In the link below Alex argues that his social network is incapable of helping him find the most interesting links.  I agree with that to some extent.   I certainly don’t think that what my social network reads is all I should read.  What would be the fun of that ?  However I do think that my social network can be a part of the filtering mechanism and help with trending topics or group related research.  Again, I think it goes back to the dynamics of relevance.  If the topic is relevant to what I’m doing and my social network is finding useful information about that topic then maybe its worth checking out.

Trust without an explicit relationship is much harder.  Once a node in the social graph is not specifically connected to another node you would be relying on someone else’s trust in a node which is tenuous at best.  However, again I would say that even a degree of trust is better than none.  If the person I have an explicit relationship with trusts this person then maybe I should too (a little)?  Obviously this doesn’t work as you get further away or you’d trust everyone, but maybe there is some measure of trust in these extended relationships that can help us discover relevant material.

I’m not sure where this will all lead over time but I do know that the current state of information overload is unsustainable and will eventually be fixed by someone, maybe you:-)

Better Feed Reading
You Have to Choose Who To Read
Fever and Future of Feed Readers

All information artifacts in the enterprise have “potential” for collaboration we just need to understand how to unleash that potential without creating another “Silo”. For all you Science Geeks out there this is very familiar to the concept of potential energy. Potential energy is the energy stored within an object waiting to be unleashed and converted to kinetic energy. Information artifacts in the enterprise possess the same potential for collaboration if the right conditions exist to release it.

Earlier in one of my posts I defined the Knowledge Cycle and its five key phases for creating and sustaining knowledge within an organization.

• Publish – information is distilled into some tangible form (document, report, wiki, email)
• Discover – finding the information from the Publish phase
• Discuss – bringing relevant parties together to better understand the information
• Personalize – adding your unique perspective to the original information
• Extend – applying newly formed knowledge to new and different scenarios

To effectively use the Knowledge Cycle and tap the potential of an organization you need to release the collaboration potential of your information using tools that understand personal and team relevance and allow information to act as the focal point of discussion and decision making. These information artifacts that have potential for collaboration we’ll call Social Objects. Social objects are information sources that encourage interaction between users due to the commonality users share with the object. They are created during the Publish phase of the Knowledge Cycle and tend to be the center around which collaboration takes place. An effective collaboration architecture allows all information sources available to a user to be considered social objects with potential for collaboration. If we use this metaphor to look across an enterprise it becomes apparent that all data sources available to knowledge workers should be considered social objects and as such collaboration should be, not only encouraged but enabled. This means that not only traditional social media like blogs, wikis, or discussion forums encourage collaboration but also that new information sources such as business intelligence reports or other business data should provide tools that bring users together and provide a platform for effective collaboration.

Let’s consider a perfectly valid business scenario that we’ve all seen dozens of times, a regional sales quota is missed in the Sales Department. The existing interactions that take place in an organization to discuss and solve this problem are broken. The current technology for finding, collaborating, and fixing the above problem relies on human interaction at nearly every level. If we walk through the current situation that most managers face it would contain the following steps:

Existing Steps

1. Get a link to a regional sales report via email at the end of the month
2. Browse the report (when you have time)
3. Discover the failing region
4. Use email to schedule an appointment among relevant parties
5. Discuss the issue at the appointed time among the parties
6. Create action items during the meeting
7. And finally repeat as necessary until you are either fired or the problem has been fixed.

In all honestly I simplified it some. It could have been worse if the manager had to go digging around to find out who the district sales manager was or use “Busy Search” to try to find a time that worked for all parties. Then there is the problem of the excessive amount of email that this flurry of activity will create and having to manage all of that!

Now consider the same scenario but this time using a smart collaboration platform. What if a platform existed that would allow us to place a report filter on the enterprise report server that would flag relevant data based on criteria we set? This platform would then discover the regional sales anomaly for us and understand that this particular object required collaboration. The platform would then automatically discover relevant parties and create a collaboration space that was focused around solving this particular issue. Busy Search is taken care of by the platform, the only thing you do is give your consent for the meeting to take place. Invitations are sent automatically. All the parties come together at the appointed time using both audio, video, and an online meeting space to discuss and archive the reasons for the missed quota. As action items arise during the meeting they are created and assigned in the meeting and associated with the initial report and workspace. As the participants begin working on their action items all relevant data is cataloged with the workspace and when the next meeting occurs the participants can easily check all the action items and progress in one place. Doesn’t that sound better?  We went from seven steps to two.

Reduced Steps:

1. Be at the meeting
2. Plan a course of action

This is the simplicity our collaboration platform needs to give us. Users shouldn’t have to worry about all the minutia involved in trying to create, discover, and manage workspaces and teams. We need tools that do the grunt work for us and let us focus on doing our REAL jobs. We need a platform that is flexible enough to allow connections to other systems for integration with communication services (email, calendaring, IM) and also provides the necessary tools that allow us to add collaboration components like comments, attachments, or tags to our Social Objects.

In the next installment we’ll look at some potential tools to make all this happen .

Which leads us to our first real question we need to answer and that is – how does collaboration add value to the bottom line of a business?  The short answer is that it doesn’t, not in and of itself.  Collaboration in the right context and under the right conditions can produce spectacular results in the form of quick customer response times, better organizational knowledge, or faster time to market.  Each of these business metrics can be enhanced through the use of collaboration tools, if done correctly, and harmed if done incorrectly.  Imagine this scenario: You thought you addressed your multi-national organization’s collaboration infrastructure needs, but it’s not looking so good. Your IT department provides many popular tools to assist your workers, such as wikis, blogs, threaded discussions, email, file shares, and more. But users are burdened with having to learn multiple applications with different user interfaces, data is often isolated in one application, users report that information is hard to find and sometimes gets lost, and none of these applications reflect your core business processes (instead, you often bend your processes to cater to the way these tools work).  This is the scenario your CIO fears most!

To begin the process of understanding how to implement collaboration software successfully in your organization, we are going to start with a high level view of how knowledge gets created in an enterprise and how collaboration tools can aid in the understanding and categorization of knowledge.

Knowledge Cycle

The diagram above represents the process information flows through as we consume and understand it.  There are 3 main phases that aid in the understanding of content and information, the first being the Publish phase.  During this phase tacit knowledge is converted into something that is explicit and consumable by others.  This phase represents the transfer of one person’s understanding, education, and wisdom into a tangible good.  The publication format can be nearly anything – a blog, wiki, document, anything that is consumable by another person.

The next phase of the cycle is the “Discover” phase.  During this period the output of the publication event is found by another person.  This can be done via search, browsing, or by someone sharing the document with you.  This is a particularly important phase because without discovery the publication event will go unused and is useless.  This area represents a significant area where technological advancement, not only in search technology but also in personal relevance, can be applied and utilized.  Services like Twitter, Digg, and TechCrunch are using community and network recommendations to provide useful metadata about discovered publications.  These services are providing a valuable filtering mechanism to help you maximize the quality of publications you consume.  RSS and Atom feeds also allow you to stay focused on publication events that occur in areas you are interested in.  The confluence of search technology, community reviews, and feeds are making the Discover phase of the Knowledge Cycle easier than ever and is providing valuable insight into what items are relevant to you.

Another interesting component of the Discover phase that has yet to be exploited is network relevance and its implications to you.  Some services such as Twitter and LinkedIn allow you to monitor a “network” for events.  LinkedIn for instance provides information about changes in your social graph as people are added and relationships forged.  This graph is a highly relevant graph to you.  It maps your closest work and personal relationships and also allows you to get a glimpse of the internetworking among your peers.  Why is this important?  Because this graph represents your preferences and social circles.  I would argue that this information is used less than it should be to help determine the publication events that we consume.  If techniques existed that would allow us to mine our social graph for not only relationships but also consumed publications and the quality of those publications, this information would be extremely helpful in shaping our information consumption.  This area of the Knowledge Cycle, I feel, will yield great results in the future in helping us discover and apply the correct information to our problem domain.

The next area of the cycle is the Discuss phase.  During this phase information has been found and consumed but may not be fully understood by the consumer.  This phase is a critical component of the Knowledge Cycle because it promotes and encourages questioning and full understanding of the material.  Many tools can be used in this phase:  phone, IM, email, forums, etc.  The requisite for the tool is that a Q&A can take place to facilitate understanding between parties.  To better disseminate information and to make it more accessible to others in the future public exchanges should be used when possible.  Communication channels like phone, IM, and email are great mechanisms that enable questioning and understanding but also limit the consumption of this “give and take” process to a very few parties.  This  sometimes is sought after behavior but in many cases the Knowledge Cycle would be more efficient if the discussions where held in public and fully searchable and cataloged with contextual elements.

“Personalize” is the phase of the cycle where the information you discovered and read is now fully understood my you, the consumer of the content.  At this point in the process you can fully appreciate the implications of the content and are beginning to understand how this new information can be applied to your own scenarios.  The collaboration events in this phase focus on your adding value to the initial publication event so others in your network can more quickly find and understand the same information.  This involves adding public tags, reviewing the content, adding Digg style ratings to the publication, or annotating the publication so that others can better understand its meaning from your perspective.  This phase adds a new dimension of information to the original content that is extremely useful especially if the contextual usage of the information is maintained either via your network or some other means.  This means that the context in which you are trying to apply the knowledge from the original publication is just as important and relevant as the value added components you associate with the document.  This context allows your network or others in similar environments to have a relevant perspective on the original information which can greatly reduce the understanding lifecycle.

The last phase of the Knowledge Cycle is the Extend phase.  In this phase we are applying the knowledge, wisdom, and education that was distilled in the original document into scenarios that we own and are a part of.  In this way knowledge has been transfered to us from someone else and used in, perhaps, totally new ways.  The context of the original information may only tangentially apply to these new scenarios, if at all.  A great example of the extension of knowledge into new areas was the seminal work by Christopher Alexander “A Pattern Language:  Towns, Buildings, Construction”.  In this book Alexander makes the case for developing construction practices based on the fundamental notion that harmonious patterns have existed in architecture for centuries and that these patterns should be reused if they add value and harmony.  This work was later applied to computer science by several researchers, most notably Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides and a new field in computer science was formed with the same basic tenets applied to computer science as Alexander had originally written about.  This example clearly demonstrates the Extend principal (although on the extreme perhaps) that allows knowledge designed for one audience and context to be discovered, discussed, and transformed into another entirely different area of study.