How Natural Language Processing Can Solve the Knowledge Retrieval Problem
Scholars, scientists, geeks, and ‘Jeopardy’ junkies are all a-Twitter over IBM’s Watson computer, the question-answering machine, that beat out two reigning Jeopardy champions. IBM scientists launched the Watson project to test whether a computing system could rival a human’s ability to answer questions posed in natural language. The results were impressive – both as a publicity stunt for IBM and in proving out the power of Natural Language Processing, but as a science, Natural Language Processing is far from new.
The explosion in unstructured text documents has presented a formidable challenge in computer-aided knowledge extraction to professionals in all fields. And simple keyword searches, advanced Boolean searches and statistically-based methods, have long been inadequate - retrieving a list of documents that might – just maybe –contain potentially relevant information.
For innovation and technical problem solving, in particular, these conventional search technologies are a failed strategy because the process of matching keywords cannot understand the context of the user’s request – their design intent and need. The result of a traditional search technologies are piles of mostly irrelevant documents, when what engineers and scientists need are answers to questions critical to their projects that require quick, informed decisions.
The reality is: a wealth of corporate wisdom resides in unstructured documents, often recorded as natural language text. This unstructured content is the mother lode of a corporation’s collective tribal wisdom – but mining it with traditional knowledge management search and navigation tools has proved disappointing to innovators who need precise insights but have very limited time. The problem is even worse when seeking relevant answers from the wealth of information on the web, in digital publications and in worldwide patent collections. Despite continual evolution of technologies to create and leverage metadata and classification schemes, researchers still find the “needles of wisdom” to be too deeply buried in the “haystacks of irrelevance” to make the effort worthwhile.
Effective knowledge retrieval demands that computers are able to correctly analyze the information requirements of the user, and to precisely match these requirements to the contents of the documents being searched. To accomplish this level of intelligence in automation, natural language text of the source documents and query must be analyzed into elements that convey meaning, and then employ these methods as the basis for unambiguous comparison between the query and the documents.
Invention Machine Goldfire, the Innovation Intelligence Platform, is powered by a unique semantic research engine that has the capability to transform unstructured documents from various electronic sources into an index that, when searched, delivers precise and relevant results. Goldfire’s Natural Language query interface enables the user to put a question in a free text format, which would be the same format as if the question were given to another person. And, once relevant knowledge has been retrieved, Goldfire presents the results in a way that makes their meaning readily apparent.
Goldfire integrates proven ideation and problem solving tools and methods with advanced knowledge retrieval capabilities to leverage relevant concepts from corporate and external content. This marriage of high-level concept extraction and problem-solving capabilities ensures engineers, scientists, researchers and other innovation workers are better informed and more creative and comprehensive in their thinking.