The highlight package contains classes to provide "keyword in context" features typically used to highlight search terms in the text of results pages.
See: Description
Interface Summary | |
---|---|
Encoder | Encodes original text. |
Formatter | Processes terms found in the original text, typically by applying some form of mark-up to highlight terms in HTML search results pages. |
Fragmenter | Implements the policy for breaking text into multiple fragments for consideration by the {@link Highlighter} class. |
Scorer | Adds to the score for a fragment based on its tokens |
Class Summary | |
---|---|
DefaultEncoder | Simple {@link Encoder} implementation that does not modify the output |
GradientFormatter | Formats text with different color intensity depending on the score of the term. |
Highlighter | Class used to markup highlighted terms found in the best sections of a text, using configurable {@link Fragmenter}, {@link Scorer}, {@link Formatter}, {@link Encoder} and tokenizers. |
NullFragmenter | {@link Fragmenter} implementation which does not fragment the text. |
QueryScorer | {@link Scorer} implementation which scores text fragments by the number of unique query terms found. |
QueryTermExtractor | Utility class used to extract the terms used in a query, plus any weights. |
SimpleFragmenter | {@link Fragmenter} implementation which breaks text up into same-size fragments with no concerns over spotting sentence boundaries. |
SimpleHTMLEncoder | Simple {@link Encoder} implementation to escape text for HTML output |
SimpleHTMLFormatter | Simple {@link Formatter} implementation to highlight terms with a pre and post tag |
SpanGradientFormatter | Formats text with different color intensity depending on the score of the term using the span tag. |
TextFragment | Low-level class used to record information about a section of a document with a score. |
TokenGroup | One, or several overlapping tokens, along with the score(s) and the scope of the original text |
TokenSources | Hides implementation issues associated with obtaining a TokenStream for use with the higlighter - can obtain from TermFreqVectors with offsets and (optionally) positions or from Analyzer class reparsing the stored content. |
WeightedTerm | Lightweight class to hold term and a weight value used for scoring this term |
IndexSearcher searcher = new IndexSearcher(ramDir); Query query = QueryParser.parse("Kenne*", FIELD_NAME, analyzer); query = query.rewrite(reader); //required to expand search terms Hits hits = searcher.search(query); Highlighter highlighter = new Highlighter(this, new QueryScorer(query)); for (int i = 0; i < hits.length(); i++) { String text = hits.doc(i).get(FIELD_NAME); TokenStream tokenStream = analyzer.tokenStream(FIELD_NAME, new StringReader(text)); // Get 3 best fragments and seperate with a "..." String result = highlighter.getBestFragments(tokenStream, text, 3, "..."); System.out.println(result); }
The highlighter takes a TokenStream as input. Until now these streams have typically been produced using an Analyzer but the new class TokenSources provides helper methods for obtaining TokenStreams from the new TermVector position support (see latest CVS version).
The new class GradientFormatter can use a scale of colors to highlight terms according to their score. A subtle use of color can help emphasise the reasons for matching (useful when doing "MoreLikeThis" queries and you want to see what the basis of the similarities are).
The QueryScorer class has a new constructor which can use an IndexReader to derive the IDF (inverse document frequency) for each term in order to influcence the score. This is useful for helping to extracting the most significant sections of a document and in supplying scores used by the new GradientFormatter to color significant words more strongly. The QueryScorer.getMaxWeight method is useful when passed to the GradientFormatter constructor to define the top score which is associated with the top color.