BlockInfo — Metadata of Memory Block
BlockInfo is a metadata of memory block (aka memory page) — the memory block’s size, the number of readers and the writer task’s id.
It has a StorageLevel, ClassTag and tellMaster flag.
Size (size attribute)
size attribute is the size of the memory block. It starts with 0.
It represents the number of bytes that BlockManager saved or BlockManager.doPutIterator.
Reader Count (readerCount attribute)
readerCount attribute is the number of readers of the memory block. It starts with 0.
It is incremented when a read lock is acquired and decreases when the following happens:
Writer Task (writerTask attribute)
writerTask attribute is the task that owns the write lock for the memory block.
A writer task can be one of the three possible identifiers:
-
NO_WRITER(i.e.-1) to denote no writers and hence no write lock in use. -
NON_TASK_WRITER(i.e.-1024) for non-task threads, e.g. by a driver thread or by unit test code. -
the task attempt id of the task which currently holds the write lock for this block.
The writer task is assigned in the following scenarios: