Package tachyon.worker

Worker process and utils for working with the worker remotely.

See: Description

Package tachyon.worker Description

Worker process and utils for working with the worker remotely. Main entry point for the worker is TachyonWorker.main(String[]) which gets started by the tachyon start scripts. The TachyonWorker class spins up the different RPC services (thrift, data) which are mostly wrappers around WorkerStorage.

Services

Thrift

The thrift service is mostly responsible for metadata operations (with a few operations that effect the worker's cached memory).

Checkpoint

The act of moving temporary data into accessible data on UnderFileSystem. This is triggered by WriteType.isThrough() operations. Implementation can be found at WorkerStorage.addCheckpoint(long, int)

Cache Block

Move's user generated blocks to the tachyon data directory. This operation expects that the caller is a local (to the node) caller, and that the input are under the user directories. Implementation can be found at WorkerStorage.cacheBlock(long, long)

Lock / Unlock

Tachyon supports cacheing blocks to local disk (client side). When this happens, a lock is given to the client to protect it from the remote block changing. Implementation can be found at WorkerStorage.lockBlock(long, long) and WorkerStorage.unlockBlock(long, long).

Data

This service is the main interaction between users and reading blocks. Currently this service only supports reading blocks (writing is to local disk). There are two different implementations of this layer: NettyDataServer and NIODataServer; netty is the default. To support both, a DataServer interface is used that just defines how to start/stop, and get port details; to start, object init is used. The current read protocol is defined in DataServerMessage. This has two different types: read DataServerMessage.createBlockRequestMessage(long, long, long) and write DataServerMessage.createBlockResponseMessage(boolean, long, long, long, ByteBuffer). Side note, the netty implementation does not use this class, but has defined two classes for the read and write case: BlockRequest, BlockResponse; theses classes are network compatible.

Copyright © 2015. All Rights Reserved.