MySolr

In one of my recent project implemented with Bobo-Browse for faceting, I ran into an issue with MultiValueFacetHandler’s 1024 values per field per record limitation.  I have some odd cases in my data set where a publication can have more 2000 authors.  This limit would stop at 1024 authors and left the rest of the authors uncredited in my search results.  It wasn’t a great loss as there are perhaps only a handful of publications with this many authors.  However, it was not great for the business so I had to create a solution.  After some searches around and tingling with the source a little, I found that by removing the hard 1024 limit was a viable solution in this particular project.  My benchmark number didn’t change after the hack and I was able to get all authors in the facet.  I can understand the hard limit was to prevent overloading the facet with too many values that could kill performance or worst, cause out of memory issue.   In fact, 1024 is a pretty high limit for a single field.  In my experience, I hardly found any multi-value field can reach anywhere near that number on a single record.  However, I just encountered such oddity.  Luckily, the number of publications with more than 1024 authors is negligible compare to the million of publications in my index.  So, this little hack didn’t produce any adverse effect for me.

Couple ways to do this hack.  One is to modify BigNestedIntArray.MAX_ITEMS directly with your desire max value.  Another way is to modify the following 3 files.

• MultiValueFacetDataCache.java
• MultiValueFacetHandler.java
• BigNestedIntArray.java

Look for the following code:

_maxItems = Math.min(maxItems, BigNestedIntArray.MAX_ITEMS);

and replace it with:

_maxItems = maxItems;

And of course, call setMaxItems on your MultiValueFacetHandler to set the desired max value.

Just want to iterate that this hack has only been tested in one particular project.  It may create instability in your project if you have a lot of records with 1024+ facet values.  I was told that the array’s key is 11 bit.  So, there is a max value at 2048.  Having more than 2048 value will likely cause an exception.  Memory consumption can also become a real issue.  Consider yourself warned.