public class CudaZeroHandler extends Object implements MemoryHandler
Modifier and Type | Field and Description |
---|---|
protected DeviceAllocationsTracker |
deviceMemoryTracker |
protected Map<Long,Integer> |
devicesAffinity |
protected NativeOps |
nativeOps |
protected AtomicLong |
zeroUseCounter |
Constructor and Description |
---|
CudaZeroHandler() |
Modifier and Type | Method and Description |
---|---|
PointersPair |
alloc(AllocationStatus targetMode,
AllocationPoint point,
AllocationShape shape,
boolean initialize)
Allocate specified memory chunk on specified device/host
|
void |
copyback(AllocationPoint point,
AllocationShape shape)
Deprecated.
|
void |
copyforward(AllocationPoint point,
AllocationShape shape)
Deprecated.
|
void |
fallback(AllocationPoint point,
AllocationShape shape)
Deprecated.
|
void |
free(AllocationPoint point,
AllocationStatus target)
This method frees memory chunk specified by pointer and location
|
long |
getAllocatedDeviceMemory(Integer device)
This method returns total amount of memory allocated at specified device
|
long |
getAllocatedDeviceObjects(Integer deviceId)
This method returns total number of object allocated on specified device
|
long |
getAllocatedHostMemory()
This method returns total amount of host memory allocated within this MemoryHandler
|
long |
getAllocatedHostObjects()
This method returns total number of allocated objects in host memory
|
long |
getAllocatedHostObjects(Long bucketId)
This method returns number of allocated objects within specific bucket
|
com.google.common.collect.Table<AllocationStatus,Integer,Long> |
getAllocationStatistics()
This method returns total amount of memory allocated within system
|
Set<Integer> |
getAvailableDevices()
This method returns set of available devices
|
ContextPool |
getContextPool() |
CudaContext |
getCudaContext()
This method returns CudaContext for current thread.
|
ExternalContext |
getDeviceContext()
This method returns ExternalContext wrapper (if applicable)
|
Integer |
getDeviceId()
This method returns CUDA deviceId for current thread
|
org.bytedeco.javacpp.Pointer |
getDeviceIdPointer()
Returns
getDeviceId() wrapped as a Pointer . |
org.bytedeco.javacpp.Pointer |
getDevicePointer(DataBuffer buffer,
CudaContext context)
PLEASE NOTE: Specific implementation, on systems without special devices can return HostPointer here
|
Set<Long> |
getDeviceTrackingPoints(Integer deviceId)
This method returns set of allocation tracking IDs for specific device
|
FlowController |
getFlowController() |
org.bytedeco.javacpp.Pointer |
getHostPointer(DataBuffer buffer)
PLEASE NOTE: This method always returns pointer within OS memory space
|
Set<Long> |
getHostTrackingPoints(Long bucketId)
This method returns sets of allocation tracking IDs for specific bucket
|
AllocationStatus |
getInitialLocation()
This method returns initial allocation location.
|
void |
init(Configuration configuration,
Allocator allocator)
This method gets called from Allocator, during Allocator/MemoryHandler initialization
|
protected void |
initCudaContextForThread(Long threadId)
This method does initialization for thread.
|
void |
initializeDevice(Long threadId,
Integer deviceId)
This method initializes specific device for current thread
|
boolean |
isDeviceDependant()
This method returns if this MemoryHandler instance is device-dependant (i.e.
|
void |
memcpy(DataBuffer dstBuffer,
DataBuffer srcBuffer)
Synchronous version of memcpy.
|
void |
memcpyAsync(DataBuffer dstBuffer,
org.bytedeco.javacpp.Pointer srcPointer,
long length,
long dstOffset)
Asynchronous version of memcpy
PLEASE NOTE: This is device-dependent method, if it's not supported in your environment, blocking call will be used instead.
|
void |
memcpyBlocking(DataBuffer dstBuffer,
org.bytedeco.javacpp.Pointer srcPointer,
long length,
long dstOffset)
Synchronous version of memcpy.
|
void |
memcpyDevice(DataBuffer dstBuffer,
org.bytedeco.javacpp.Pointer srcPointer,
long length,
long dstOffset,
CudaContext context) |
void |
memcpySpecial(DataBuffer dstBuffer,
org.bytedeco.javacpp.Pointer srcPointer,
long length,
long dstOffset)
Special memcpy version, addressing shapeInfoDataBuffer copies
PLEASE NOTE: Blocking H->H, Async H->D
|
boolean |
pingDeviceForFreeMemory(Integer deviceId,
long requiredMemory)
This method checks if specified device has free memory
|
boolean |
promoteObject(DataBuffer buffer)
This method moves specific object from zero-copy memory to device memory
PLEASE NOTE: DO NOT EVER USE THIS METHOD MANUALLY, UNLESS YOU 100% HAVE TO
|
void |
purgeDeviceObject(Long threadId,
Integer deviceId,
Long objectId,
AllocationPoint point,
boolean copyback)
This method explicitly removes object from device memory.
|
void |
purgeZeroObject(Long bucketId,
Long objectId,
AllocationPoint point,
boolean copyback)
This method explicitly removes object from zero-copy memory.
|
void |
registerAction(CudaContext context,
INDArray result,
INDArray... operands) |
void |
relocate(AllocationStatus currentStatus,
AllocationStatus targetStatus,
AllocationPoint point,
AllocationShape shape,
CudaContext context)
Copies specific chunk of memory from one storage to another
Possible directions: HOST -> DEVICE, DEVICE -> HOST
|
void |
relocateObject(DataBuffer buffer) |
void |
synchronizeThreadDevice(Long threadId,
Integer deviceId,
AllocationPoint point)
This method causes memory synchronization on host side.
|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getMemoryProvider
protected final AtomicLong zeroUseCounter
protected volatile DeviceAllocationsTracker deviceMemoryTracker
protected NativeOps nativeOps
public void init(@NonNull Configuration configuration, @NonNull Allocator allocator)
init
in interface MemoryHandler
configuration
- allocator
- public PointersPair alloc(AllocationStatus targetMode, AllocationPoint point, AllocationShape shape, boolean initialize)
alloc
in interface MemoryHandler
targetMode
- valid arguments are DEVICE, ZEROshape
- public boolean pingDeviceForFreeMemory(Integer deviceId, long requiredMemory)
pingDeviceForFreeMemory
in interface MemoryHandler
deviceId
- requiredMemory
- public void relocate(AllocationStatus currentStatus, AllocationStatus targetStatus, AllocationPoint point, AllocationShape shape, CudaContext context)
relocate
in interface MemoryHandler
currentStatus
- targetStatus
- point
- @Deprecated public void copyback(AllocationPoint point, AllocationShape shape)
copyback
in interface MemoryHandler
point
- @Deprecated public void copyforward(AllocationPoint point, AllocationShape shape)
copyforward
in interface MemoryHandler
point
- @Deprecated public void fallback(AllocationPoint point, AllocationShape shape)
fallback
in interface MemoryHandler
point
- shape
- public void free(AllocationPoint point, AllocationStatus target)
free
in interface MemoryHandler
point
- Pointerpublic AllocationStatus getInitialLocation()
getInitialLocation
in interface MemoryHandler
public void initializeDevice(Long threadId, Integer deviceId)
initializeDevice
in interface MemoryHandler
threadId
- deviceId
- public void memcpyAsync(DataBuffer dstBuffer, org.bytedeco.javacpp.Pointer srcPointer, long length, long dstOffset)
memcpyAsync
in interface MemoryHandler
dstBuffer
- srcPointer
- length
- dstOffset
- public void memcpyDevice(DataBuffer dstBuffer, org.bytedeco.javacpp.Pointer srcPointer, long length, long dstOffset, CudaContext context)
memcpyDevice
in interface MemoryHandler
public void memcpySpecial(DataBuffer dstBuffer, org.bytedeco.javacpp.Pointer srcPointer, long length, long dstOffset)
memcpySpecial
in interface MemoryHandler
dstBuffer
- srcPointer
- length
- dstOffset
- public void memcpyBlocking(DataBuffer dstBuffer, org.bytedeco.javacpp.Pointer srcPointer, long length, long dstOffset)
memcpyBlocking
in interface MemoryHandler
dstBuffer
- srcPointer
- length
- dstOffset
- public void memcpy(DataBuffer dstBuffer, DataBuffer srcBuffer)
memcpy
in interface MemoryHandler
dstBuffer
- srcBuffer
- public org.bytedeco.javacpp.Pointer getDevicePointer(DataBuffer buffer, CudaContext context)
getDevicePointer
in interface MemoryHandler
buffer
- public org.bytedeco.javacpp.Pointer getHostPointer(DataBuffer buffer)
getHostPointer
in interface MemoryHandler
buffer
- public void relocateObject(DataBuffer buffer)
relocateObject
in interface MemoryHandler
public boolean promoteObject(DataBuffer buffer)
promoteObject
in interface MemoryHandler
public com.google.common.collect.Table<AllocationStatus,Integer,Long> getAllocationStatistics()
getAllocationStatistics
in interface MemoryHandler
public long getAllocatedDeviceMemory(Integer device)
getAllocatedDeviceMemory
in interface MemoryHandler
device
- public long getAllocatedHostMemory()
getAllocatedHostMemory
in interface MemoryHandler
public long getAllocatedDeviceObjects(Integer deviceId)
getAllocatedDeviceObjects
in interface MemoryHandler
deviceId
- public long getAllocatedHostObjects(Long bucketId)
getAllocatedHostObjects
in interface MemoryHandler
bucketId
- public long getAllocatedHostObjects()
getAllocatedHostObjects
in interface MemoryHandler
public Set<Long> getDeviceTrackingPoints(Integer deviceId)
getDeviceTrackingPoints
in interface MemoryHandler
deviceId
- public Set<Long> getHostTrackingPoints(Long bucketId)
getHostTrackingPoints
in interface MemoryHandler
bucketId
- public void purgeDeviceObject(Long threadId, Integer deviceId, Long objectId, AllocationPoint point, boolean copyback)
purgeDeviceObject
in interface MemoryHandler
threadId
- objectId
- copyback
- if TRUE, corresponding memory block on JVM side will be updated, if FALSE - memory will be just discardedpublic void purgeZeroObject(Long bucketId, Long objectId, AllocationPoint point, boolean copyback)
purgeZeroObject
in interface MemoryHandler
bucketId
- objectId
- copyback
- if TRUE, corresponding memory block on JVM side will be updated, if FALSE - memory will be just discardedpublic Integer getDeviceId()
getDeviceId
in interface MemoryHandler
public org.bytedeco.javacpp.Pointer getDeviceIdPointer()
getDeviceId()
wrapped as a Pointer
.getDeviceIdPointer
in interface MemoryHandler
public Set<Integer> getAvailableDevices()
getAvailableDevices
in interface MemoryHandler
public ExternalContext getDeviceContext()
getDeviceContext
in interface MemoryHandler
public CudaContext getCudaContext()
protected void initCudaContextForThread(Long threadId)
threadId
- public boolean isDeviceDependant()
isDeviceDependant
in interface MemoryHandler
public void synchronizeThreadDevice(Long threadId, Integer deviceId, AllocationPoint point)
synchronizeThreadDevice
in interface MemoryHandler
threadId
- deviceId
- point
- public void registerAction(CudaContext context, INDArray result, INDArray... operands)
registerAction
in interface MemoryHandler
public FlowController getFlowController()
getFlowController
in interface MemoryHandler
public ContextPool getContextPool()
getContextPool
in interface MemoryHandler
Copyright © 2016. All Rights Reserved.