Package org.apache.lucene.index
Class ConcurrentMergeScheduler
- java.lang.Object
-
- org.apache.lucene.index.MergeScheduler
-
- org.apache.lucene.index.ConcurrentMergeScheduler
-
public class ConcurrentMergeScheduler extends MergeScheduler
AMergeScheduler
that runs each merge using a separate thread.Specify the max number of threads that may run at once with
setMaxThreadCount(int)
.Separately specify the maximum number of simultaneous merges with
setMaxMergeCount(int)
. If the number of merges exceeds the max number of threads then the largest merges are paused until one of the smaller merges completes.If more than
getMaxMergeCount()
merges are requested then this class will forcefully throttle the incoming threads by pausing until one more more merges complete.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected class
ConcurrentMergeScheduler.MergeThread
-
Field Summary
Fields Modifier and Type Field Description protected static Comparator<ConcurrentMergeScheduler.MergeThread>
compareByMergeDocCount
protected Directory
dir
protected int
mergeThreadCount
protected List<ConcurrentMergeScheduler.MergeThread>
mergeThreads
protected IndexWriter
writer
-
Constructor Summary
Constructors Constructor Description ConcurrentMergeScheduler()
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Deprecated Methods Modifier and Type Method Description static boolean
anyUnhandledExceptions()
Used for testingstatic void
clearUnhandledExceptions()
void
close()
Close this MergeScheduler.protected void
doMerge(MergePolicy.OneMerge merge)
Does the actual merge, by callingIndexWriter.merge(org.apache.lucene.index.MergePolicy.OneMerge)
int
getMaxMergeCount()
int
getMaxThreadCount()
protected ConcurrentMergeScheduler.MergeThread
getMergeThread(IndexWriter writer, MergePolicy.OneMerge merge)
Create and return a new MergeThreadint
getMergeThreadPriority()
Return the priority that merge threads run at.protected void
handleMergeException(Throwable exc)
Called when an exception is hit in a background merge threadvoid
merge(IndexWriter writer)
Run the merges provided byIndexWriter.getNextMerge()
.protected int
mergeThreadCount()
Returns the number of merge threads that are alive.protected void
message(String message)
Outputs the given message - this method assumesverbose()
was called and returned true.void
setMaxMergeCount(int count)
Sets the max # simultaneous merges that are allowed.void
setMaxThreadCount(int count)
Sets the max # simultaneous merge threads that should be running at once.void
setMergeThreadPriority(int pri)
Set the base priority that merge threads run at.static void
setTestMode()
Deprecated.this test mode code will be removed in a future releasevoid
sync()
Wait for any running merge threads to finishprotected void
updateMergeThreads()
Called whenever the running merges have changed, to pause & unpause threads.protected boolean
verbose()
Returns true if verbosing is enabled.
-
-
-
Field Detail
-
mergeThreads
protected List<ConcurrentMergeScheduler.MergeThread> mergeThreads
-
dir
protected Directory dir
-
writer
protected IndexWriter writer
-
mergeThreadCount
protected int mergeThreadCount
-
compareByMergeDocCount
protected static final Comparator<ConcurrentMergeScheduler.MergeThread> compareByMergeDocCount
-
-
Method Detail
-
setMaxThreadCount
public void setMaxThreadCount(int count)
Sets the max # simultaneous merge threads that should be running at once. This must be <=setMaxMergeCount(int)
.
-
getMaxThreadCount
public int getMaxThreadCount()
- See Also:
setMaxThreadCount(int)
-
setMaxMergeCount
public void setMaxMergeCount(int count)
Sets the max # simultaneous merges that are allowed. If a merge is necessary yet we already have this many threads running, the incoming thread (that is calling add/updateDocument) will block until a merge thread has completed. Note that we will only run the smallestsetMaxThreadCount(int)
merges at a time.
-
getMaxMergeCount
public int getMaxMergeCount()
-
getMergeThreadPriority
public int getMergeThreadPriority()
Return the priority that merge threads run at. By default the priority is 1 plus the priority of (ie, slightly higher priority than) the first thread that calls merge.
-
setMergeThreadPriority
public void setMergeThreadPriority(int pri)
Set the base priority that merge threads run at. Note that CMS may increase priority of some merge threads beyond this base priority. It's best not to set this any higher than Thread.MAX_PRIORITY-maxThreadCount, so that CMS has room to set relative priority among threads.
-
updateMergeThreads
protected void updateMergeThreads()
Called whenever the running merges have changed, to pause & unpause threads. This method sorts the merge threads by their merge size in descending order and then pauses/unpauses threads from first to last -- that way, smaller merges are guaranteed to run before larger ones.
-
verbose
protected boolean verbose()
Returns true if verbosing is enabled. This method is usually used in conjunction withmessage(String)
, like that:if (verbose()) { message("your message"); }
-
message
protected void message(String message)
Outputs the given message - this method assumesverbose()
was called and returned true.
-
close
public void close()
Description copied from class:MergeScheduler
Close this MergeScheduler.- Specified by:
close
in classMergeScheduler
-
sync
public void sync()
Wait for any running merge threads to finish
-
mergeThreadCount
protected int mergeThreadCount()
Returns the number of merge threads that are alive. Note that this number is ≤mergeThreads
size.
-
merge
public void merge(IndexWriter writer) throws IOException
Description copied from class:MergeScheduler
Run the merges provided byIndexWriter.getNextMerge()
.- Specified by:
merge
in classMergeScheduler
- Throws:
IOException
-
doMerge
protected void doMerge(MergePolicy.OneMerge merge) throws IOException
Does the actual merge, by callingIndexWriter.merge(org.apache.lucene.index.MergePolicy.OneMerge)
- Throws:
IOException
-
getMergeThread
protected ConcurrentMergeScheduler.MergeThread getMergeThread(IndexWriter writer, MergePolicy.OneMerge merge) throws IOException
Create and return a new MergeThread- Throws:
IOException
-
handleMergeException
protected void handleMergeException(Throwable exc)
Called when an exception is hit in a background merge thread
-
anyUnhandledExceptions
public static boolean anyUnhandledExceptions()
Used for testing
-
clearUnhandledExceptions
public static void clearUnhandledExceptions()
-
setTestMode
@Deprecated public static void setTestMode()
Deprecated.this test mode code will be removed in a future release
-
-