Package org.apache.bookkeeper.bookie

Class Journal

java.lang.Object

java.lang.Thread

io.netty.util.concurrent.FastThreadLocalThread

org.apache.bookkeeper.bookie.BookieThread

org.apache.bookkeeper.bookie.BookieCriticalThread

org.apache.bookkeeper.bookie.Journal

All Implemented Interfaces:

java.lang.Runnable, java.lang.Thread.UncaughtExceptionHandler, CheckpointSource

public class Journal
extends BookieCriticalThread
implements CheckpointSource

Provide journal related management.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	Journal.BufferedChannelBuilder	For testability.
class	Journal.ForceWriteRequest	Token which represents the need to force a write to the Journal.
static interface	Journal.JournalIdFilter	Filter to pickup journals.
static interface	Journal.JournalScanner	Scanner used to scan a journal.
class	Journal.LastLogMark	Last Log Mark.
(package private) static class	Journal.QueueEntry	Journal Entry to Record.

Nested classes/interfaces inherited from class java.lang.Thread

java.lang.Thread.State, java.lang.Thread.UncaughtExceptionHandler

Nested classes/interfaces inherited from interface org.apache.bookkeeper.bookie.CheckpointSource

CheckpointSource.Checkpoint

Field Summary

Fields

Modifier and Type	Field	Description
(package private) ServerConfiguration	conf
(package private) FileChannelProvider	fileChannelProvider
(package private) java.util.concurrent.BlockingQueue<Journal.ForceWriteRequest>	forceWriteRequests
(package private) org.apache.bookkeeper.bookie.Journal.ForceWriteThread	forceWriteThread
(package private) java.io.File	journalDirectory
(package private) long	journalPreAllocSize
(package private) int	journalWriteBufferSize
(package private) static int	KB
(package private) int	maxBackupJournals
(package private) long	maxJournalSize
(package private) static long	MB
(package private) static int	PADDING_MASK
(package private) java.util.concurrent.BlockingQueue<Journal.QueueEntry>	queue
(package private) boolean	running

Fields inherited from class java.lang.Thread

MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY

Fields inherited from interface org.apache.bookkeeper.bookie.CheckpointSource

DEFAULT

Constructor Summary

Constructors

Constructor	Description
Journal(int journalIndex, java.io.File journalDirectory, ServerConfiguration conf, LedgerDirsManager ledgerDirsManager)
Journal(int journalIndex, java.io.File journalDirectory, ServerConfiguration conf, LedgerDirsManager ledgerDirsManager, StatsLogger statsLogger, io.netty.buffer.ByteBufAllocator allocator)

Method Summary

Modifier and Type	Method	Description
void	checkpointComplete(CheckpointSource.Checkpoint checkpoint, boolean compact)	Telling journal a checkpoint is finished.
(package private) void	forceLedger(long ledgerId, BookkeeperInternalCallbacks.WriteCallback cb, java.lang.Object ctx)
Journal.BufferedChannelBuilder	getBufferedChannelBuilder()
java.io.File	getJournalDirectory()
int	getJournalQueueLength()	Get the length of journal entries queue.
(package private) JournalStats	getJournalStats()
Journal.LastLogMark	getLastLogMark()
(package private) long	getMemoryUsage()
void	joinThread()	Wait for the Journal thread to exit.
static java.util.List<java.lang.Long>	listJournalIds(java.io.File journalDir, Journal.JournalIdFilter filter)	List all journal ids by a specified journal id filer.
void	logAddEntry(long ledgerId, long entryId, io.netty.buffer.ByteBuf entry, boolean ackBeforeSync, BookkeeperInternalCallbacks.WriteCallback cb, java.lang.Object ctx)
void	logAddEntry(io.netty.buffer.ByteBuf entry, boolean ackBeforeSync, BookkeeperInternalCallbacks.WriteCallback cb, java.lang.Object ctx)	record an add entry operation in journal.
void	logAddEntry(java.nio.ByteBuffer entry, boolean ackBeforeSync, BookkeeperInternalCallbacks.WriteCallback cb, java.lang.Object ctx)
CheckpointSource.Checkpoint	newCheckpoint()	Application tried to schedule a checkpoint.
void	run()	A thread used for persisting journal entries to journal files.
long	scanJournal(long journalId, long journalPos, Journal.JournalScanner scanner)	Scan the journal.
(package private) void	setLastLogMark(java.lang.Long id, long scanOffset)	Update lastLogMark of the journal Indicates that the file has been processed.
void	shutdown()	Shuts down the journal.
(package private) static void	writePaddingBytes(JournalChannel jc, io.netty.buffer.ByteBuf paddingBuffer, int journalAlignSize)

Methods inherited from class org.apache.bookkeeper.bookie.BookieCriticalThread

handleException

Methods inherited from class org.apache.bookkeeper.bookie.BookieThread

uncaughtException

Methods inherited from class io.netty.util.concurrent.FastThreadLocalThread

setThreadLocalMap, threadLocalMap, willCleanupFastThreadLocals, willCleanupFastThreadLocals

Methods inherited from class java.lang.Thread

activeCount, checkAccess, clone, countStackFrames, currentThread, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, onSpinWait, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, suspend, toString, yield

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

PADDING_MASK

static final int PADDING_MASK

See Also:

Constant Field Values

MB

static final long MB

See Also:

Constant Field Values

KB

static final int KB

See Also:

Constant Field Values

maxJournalSize

final long maxJournalSize

journalPreAllocSize

final long journalPreAllocSize

journalWriteBufferSize

final int journalWriteBufferSize

maxBackupJournals

final int maxBackupJournals

journalDirectory

final java.io.File journalDirectory

conf

final ServerConfiguration conf

forceWriteThread

final org.apache.bookkeeper.bookie.Journal.ForceWriteThread forceWriteThread

fileChannelProvider

final FileChannelProvider fileChannelProvider

queue

final java.util.concurrent.BlockingQueue<Journal.QueueEntry> queue

forceWriteRequests

final java.util.concurrent.BlockingQueue<Journal.ForceWriteRequest> forceWriteRequests

running

volatile boolean running

Constructor Detail

Journal

public Journal(int journalIndex,
               java.io.File journalDirectory,
               ServerConfiguration conf,
               LedgerDirsManager ledgerDirsManager)

Journal

public Journal(int journalIndex,
               java.io.File journalDirectory,
               ServerConfiguration conf,
               LedgerDirsManager ledgerDirsManager,
               StatsLogger statsLogger,
               io.netty.buffer.ByteBufAllocator allocator)

Method Detail

listJournalIds

public static java.util.List<java.lang.Long> listJournalIds(java.io.File journalDir,
                                                            Journal.JournalIdFilter filter)

List all journal ids by a specified journal id filer.

Parameters:

journalDir - journal dir

filter - journal id filter

Returns:

list of filtered ids

writePaddingBytes

static void writePaddingBytes(JournalChannel jc,
                              io.netty.buffer.ByteBuf paddingBuffer,
                              int journalAlignSize)
                       throws java.io.IOException

Throws:

java.io.IOException

getJournalStats

JournalStats getJournalStats()

getJournalDirectory

public java.io.File getJournalDirectory()

getLastLogMark

public Journal.LastLogMark getLastLogMark()

setLastLogMark

void setLastLogMark(java.lang.Long id,
                    long scanOffset)

Update lastLogMark of the journal Indicates that the file has been processed.

Parameters:

id -

scanOffset -

newCheckpoint

public CheckpointSource.Checkpoint newCheckpoint()

Application tried to schedule a checkpoint. After all the txns added before checkpoint are persisted, a checkpoint will be returned to application. Application could use checkpoint to do its logic.

Specified by:

newCheckpoint in interface CheckpointSource

Returns:

checkpoint.

checkpointComplete

public void checkpointComplete(CheckpointSource.Checkpoint checkpoint,
                               boolean compact)
                        throws java.io.IOException

Telling journal a checkpoint is finished.

Specified by:

checkpointComplete in interface CheckpointSource

Parameters:

checkpoint - The checkpoint that has been completed

compact - Flag to compact old checkpoints.

Throws:

java.io.IOException

scanJournal

public long scanJournal(long journalId,
                        long journalPos,
                        Journal.JournalScanner scanner)
                 throws java.io.IOException

Scan the journal.

Parameters:

journalId - Journal Log Id

journalPos - Offset to start scanning

scanner - Scanner to handle entries

Returns:

scanOffset - represents the byte till which journal was read

Throws:

java.io.IOException

logAddEntry

public void logAddEntry(java.nio.ByteBuffer entry,
                        boolean ackBeforeSync,
                        BookkeeperInternalCallbacks.WriteCallback cb,
                        java.lang.Object ctx)
                 throws java.lang.InterruptedException

Throws:

java.lang.InterruptedException

logAddEntry

public void logAddEntry(io.netty.buffer.ByteBuf entry,
                        boolean ackBeforeSync,
                        BookkeeperInternalCallbacks.WriteCallback cb,
                        java.lang.Object ctx)
                 throws java.lang.InterruptedException

record an add entry operation in journal.

Throws:

java.lang.InterruptedException

logAddEntry

public void logAddEntry(long ledgerId,
                        long entryId,
                        io.netty.buffer.ByteBuf entry,
                        boolean ackBeforeSync,
                        BookkeeperInternalCallbacks.WriteCallback cb,
                        java.lang.Object ctx)
                 throws java.lang.InterruptedException

Throws:

java.lang.InterruptedException

forceLedger

void forceLedger(long ledgerId,
                 BookkeeperInternalCallbacks.WriteCallback cb,
                 java.lang.Object ctx)

getJournalQueueLength

public int getJournalQueueLength()

Get the length of journal entries queue.

Returns:

length of journal entry queue.

run

public void run()

A thread used for persisting journal entries to journal files.

Besides persisting journal entries, it also takes responsibility of rolling journal files when a journal file reaches journal file size limitation.

During journal rolling, it first closes the writing journal, generates new journal file using current timestamp, and continue persistence logic. Those journals will be garbage collected in SyncThread.

Specified by:

run in interface java.lang.Runnable

Overrides:

run in class java.lang.Thread

See Also:

SyncThread

getBufferedChannelBuilder

public Journal.BufferedChannelBuilder getBufferedChannelBuilder()

shutdown

public void shutdown()

Shuts down the journal.

joinThread

public void joinThread()
                throws java.lang.InterruptedException

Wait for the Journal thread to exit. This is method is needed in order to mock the journal, we can't mock final method of java.lang.Thread class

Throws:

java.lang.InterruptedException

getMemoryUsage

long getMemoryUsage()