-- Hoogle documentation, generated by Haddock
-- See Hoogle, http://www.haskell.org/hoogle/

-- | In memory storage of deeply evaluated data structure
--   
--   This package provides minimal functionality for working with "compact
--   regions", which hold a fully evaluated Haskell object graph. These
--   regions maintain the invariant that no pointers live inside the struct
--   that point outside it, which ensures efficient garbage collection
--   without ever reading the structure contents (effectively, it works as
--   a manually managed "oldest generation" which is never freed until the
--   whole is released). Internally, the struct is stored a single
--   contiguous block of memory, which allows efficient serialization and
--   deserialization of structs for distributed computing. This package
--   provides a low-level API; see also the <packagecompact
--   compact package> which provides a user-facing API.
@package ghc-compact
@version 0.1.0.0

-- | This module provides a data structure, called a Compact, for
--   holding immutable, fully evaluated data in a consecutive block of
--   memory. Compact regions are good for two things:
--   
--   
--   
- Data in a compact region is not traversed during GC; any incoming
--   pointer to a compact region keeps the entire region live. Thus, if you
--   put a long-lived data structure in a compact region, you may save a
--   lot of cycles during major collections, since you will no longer be
--   (uselessly) retraversing this data structure.
--   
- Because the data is stored contiguously, you can easily dump the
--   memory to disk and/or send it over the network. For applications that
--   are not bandwidth bound (GHC's heap representation can be as much of a
--   x4 expansion over a binary serialization), this can lead to
--   substantial speedups.
--   
--   
--   For example, suppose you have a function loadBigStruct :: IO
--   BigStruct, which loads a large data structure from the file
--   system. You can "compact" the structure with the following code:
--   
--   
--   do r <- compact =<< loadBigStruct
--      let x = getCompact r :: BigStruct
--      -- Do things with x
--   
--   
--   Note that compact will not preserve internal sharing; use
--   compactWithSharing (which is 10x slower) if you have cycles
--   and/or must preserve sharing. The Compact pointer r
--   can be used to add more data to a compact region; see
--   compactAdd or compactAddWithSharing.
--   
--   The implementation of compact regions is described by:
--   
--   
--   
- Edward Z. Yang, Giovanni Campagna, Ömer Ağacan, Ahmed El-Hassany,
--   Abhishek Kulkarni, Ryan Newton. "/Efficient communication and
--   Collection with Compact Normal Forms/". In Proceedings of the 20th ACM
--   SIGPLAN International Conference on Functional Programming. September
--   2015. http://ezyang.com/compact.html
--   
--   
--   This library is supported by GHC 8.2 and later.
module GHC.Compact

-- | A Compact contains fully evaluated, pure, immutable data.
--   
--   Compact serves two purposes:
--   
--   
--   
- Data stored in a Compact has no garbage collection
--   overhead. The garbage collector considers the whole Compact to
--   be alive if there is a reference to any object within it.
--   
- A Compact can be serialized, stored, and deserialized
--   again. The serialized data can only be deserialized by the exact
--   binary that created it, but it can be stored indefinitely before
--   deserialization.
--   
--   
--   Compacts are self-contained, so compacting data involves copying it;
--   if you have data that lives in two Compacts, each will have a
--   separate copy of the data.
--   
--   The cost of compaction is fully evaluating the data + copying it.
--   However, because compact does not stop-the-world, retaining
--   internal sharing during the compaction process is very costly. The
--   user can choose whether to compact or
--   compactWithSharing.
--   
--   When you have a Compact a, you can get a pointer to
--   the actual object in the region using getCompact. The
--   Compact type serves as handle on the region itself; you can use
--   this handle to add data to a specific Compact with
--   compactAdd or compactAddWithSharing (giving you a new
--   handle which corresponds to the same compact region, but points to the
--   newly added object in the region). At the moment, due to technical
--   reasons, it's not possible to get the Compact a if you
--   only have an a, so make sure you hold on to the handle as
--   necessary.
--   
--   Data in a compact doesn't ever move, so compacting data is also a way
--   to pin arbitrary data structures in memory.
--   
--   There are some limitations on what can be compacted:
--   
--   
--   
- Functions. Compaction only applies to data.
--   
- Pinned ByteArray# objects cannot be compacted. This is for
--   a good reason: the memory is pinned so that it can be referenced by
--   address (the address might be stored in a C data structure, for
--   example), so we can't make a copy of it to store in the
--   Compact.
--   
- Objects with mutable pointer fields (e.g. IORef,
--   MutableArray) also cannot be compacted, because subsequent
--   mutation would destroy the property that a compact is
--   self-contained.
--   
--   
--   If compaction encounters any of the above, a CompactionFailed
--   exception will be thrown by the compaction operation.
data Compact a
Compact :: Compact# -> a -> !MVar () -> Compact a

-- | Compact a value. O(size of unshared data)
--   
--   If the structure contains any internal sharing, the shared data will
--   be duplicated during the compaction process. This will not terminate
--   if the structure contains cycles (use compactWithSharing
--   instead).
--   
--   The object in question must not contain any functions or data with
--   mutable pointers; if it does, compact will raise an exception.
--   In the future, we may add a type class which will help statically
--   check if this is the case or not.
compact :: a -> IO (Compact a)

-- | Compact a value, retaining any internal sharing and cycles. O(size
--   of data)
--   
--   This is typically about 10x slower than compact, because it
--   works by maintaining a hash table mapping uncompacted objects to
--   compacted objects.
--   
--   The object in question must not contain any functions or data with
--   mutable pointers; if it does, compact will raise an exception.
--   In the future, we may add a type class which will help statically
--   check if this is the case or not.
compactWithSharing :: a -> IO (Compact a)

-- | Add a value to an existing Compact. This will help you avoid
--   copying when the value contains pointers into the compact region, but
--   remember that after compaction this value will only be deallocated
--   with the entire compact region.
--   
--   Behaves exactly like compact with respect to sharing and what
--   data it accepts.
compactAdd :: Compact b -> a -> IO (Compact a)

-- | Add a value to an existing Compact, like compactAdd, but
--   behaving exactly like compactWithSharing with respect to
--   sharing and what data it accepts.
compactAddWithSharing :: Compact b -> a -> IO (Compact a)

-- | Retrieve a direct pointer to the value pointed at by a Compact
--   reference. If you have used compactAdd, there may be multiple
--   Compact references into the same compact region. Upholds the
--   property:
--   
--   
--   inCompact c (getCompact c) == True
--   
getCompact :: Compact a -> a

-- | Check if the second argument is inside the passed Compact.
inCompact :: Compact b -> a -> IO Bool

-- | Check if the argument is in any Compact. If true, the value in
--   question is also fully evaluated, since any value in a compact region
--   must be fully evaluated.
isCompact :: a -> IO Bool

-- | Returns the size in bytes of the compact region.
compactSize :: Compact a -> IO Word

-- | Experimental This function doesn't actually resize a compact
--   region; rather, it changes the default block size which we allocate
--   when the current block runs out of space, and also appends a block to
--   the compact region.
compactResize :: Compact a -> Word -> IO ()

-- | Make a new Compact object, given a pointer to the true
--   underlying region. You must uphold the invariant that a lives
--   in the compact region.
mkCompact :: Compact# -> a -> State# RealWorld -> (# State# RealWorld, Compact a #)

-- | Transfer a into a new compact region, with a preallocated
--   size (in bytes), possibly preserving sharing or not. If you know how
--   big the data structure in question is, you can save time by picking an
--   appropriate block size for the compact region.
compactSized :: Int -> Bool -> a -> IO (Compact a)

-- | This module contains support for serializing a Compact for network
--   transmission and on-disk storage.
--   
--   Since: 1.0.0
module GHC.Compact.Serialized

-- | A serialized version of the Compact metadata (each block with
--   address and size and the address of the root). This structure is meant
--   to be sent alongside the actual Compact data. It can be sent
--   out of band in advance if the data is to be sent over RDMA (which
--   requires both sender and receiver to have pinned buffers).
data SerializedCompact a
SerializedCompact :: [(Ptr (), Word)] -> Ptr () -> SerializedCompact a
[serializedCompactBlockList] :: SerializedCompact a -> [(Ptr (), Word)]
[serializedCompactRoot] :: SerializedCompact a -> Ptr ()

-- | Serialize the Compact, and call the provided function with with
--   the Compact serialized representation. It is not safe to return
--   the pointer from the action and use it after the action completes: all
--   uses must be inside this bracket, since we cannot guarantee that the
--   compact region will stay live from the Ptr object. For example,
--   it would be unsound to use unsafeInterleaveIO to lazily
--   construct a lazy bytestring from the Ptr.
withSerializedCompact :: Compact a -> (SerializedCompact a -> IO c) -> IO c

-- | Deserialize a SerializedCompact into a in-memory
--   Compact. The provided function will be called with the address
--   and size of each newly allocated block in succession, and should fill
--   the memory from the external source (eg. by reading from a socket or
--   from disk) importCompact can return Nothing if the
--   Compact was corrupt or it had pointers that could not be
--   adjusted.
importCompact :: SerializedCompact a -> (Ptr b -> Word -> IO ()) -> IO (Maybe (Compact a))

-- | Convenience function for importing a compact region that is
--   represented by a list of strict ByteStrings.
importCompactByteStrings :: SerializedCompact a -> [ByteString] -> IO (Maybe (Compact a))
