Nebulostore Features assigned to development iterations

First iteration (completed, mid-Feb 2012)

  • internal architecture
  • wrapping jxta inside CommunicationModule
  • API: get object (based on physical address)
  • API: dht put/get
  • bdbDht: a key-value base stored entirely on a single peer

Tag 0.1 - completed first iteration, before introduction of object API.

Second iteration (completed, mid-Mar 2012)

  • ObjectAPI: remaining functions (Bolek) done
    • file read & write (Bolek) done
    • lists, ordering, consistency (Bolek) done
  • top-level DHT (Marcin) done
    • no object removal
    • write-only DHT for PeerSoN
  • asynchronous messages (Bolek+Szymon) done
  • trivial broker (Bolek) done
  • metadata for files (Bolek) done
    • simple metadata: a file with one chunk
  • planet-lab tests (Marcin) done
  • test environment (Szymon) done
    • creating a network configuration
    • simulating failures
    • measurements

Third iteration (completed, May 2012)

  • propagation of objects' updates / new files (Szymon) done
    • new module needed
    • consistency module
    • roles of creator and replicating peer
  • list ordering and consistency (Bolek) done
  • replicator Szymon (done)
  • big files span multiple blocks; individual block is a NebuloObject (class FileChunk) (Bolek) done
  • distributed tests with transient failures (Szymon + Marcin) (done, Marcin's changes to be merged with trunk)
    • asynch messages
    • update propagation
  • delete & truncate (Bolek) done
  • periodic reads of asynchronous messages (Szymon) (done)
  • debug JXTA communication layer: scale up to 100 peers (Marcin) (not done, problems)
  • memory leaks? (Marcin) (no memory leaks)
  • dirty/clean files for replicas that came on-line, know that a file has been updated, but can't access the new version (Szymon) done
  • detecting conflicting changes in files (Szymon) done
  • data search (DQL, Marcin) (done)

Tag 0.3 - completed third iteration, before JXTA replacement

Fourth Iteration (completed, early February 2013)

  • network communication library including UPnP (Grzesiek) done
  • maven integration (Rafał) done
  • subscriptions and notifications (Rafał) done
  • improving unit test coverage (Bolek) (4 tests done)
  • conductor to easily perform planet-lab tests (Bolek) done
  • configuration of modules by dependency injection (Rafał, Bolek) done
  • single main class (Bolek) done
  • main configuration file (Bolek) done
  • peer discovery: BRAHMS gossiping (Grzesiek) done
  • removing ANT (Bolek) done
  • moving system tests to main (Bolek,Grzesiek) done
  • read-write tests on 10+ Planet-Lab peers (Bolek) done

Tag 0.4 - completed fourth iteration: new communication layer, Maven, automatic Planet-Lab tests.

Fifth Iteration (completed, April 2013)

  • debug lists (Bolek) (done)
  • scripts for parallel PlanetLab deployment (Bolek) (done)
  • list performance confirmed on ~100 peers (Bolek) (done)
  • simple GUI: store file, read file, list add element, list read (Jadzia) (done)
  • bug in network module (Grzesiek) (done)
  • remove instanceID (Bolek, Grzesiek checks), commAddress has to stay (done)
  • pseudo-tests (programs) in main repository (Bolek) (done)
  • remove special jobId for the persistent broker (Bolek) (done)

teration goal: Easy replication of a local directory with files using GUI. Directory stored as a NebuloList and replicated on ~5 peers selected by Broker from a total of ~15 peers.

Sixth iteration (June 2013)

  • Reflection-based visitor (Bolek) (done)
  • Persisting metadata in Replicator (Bolek) (done)
  • Lower package entanglement (Bolek)
    • cleanup inside appcore
    • separate config for CommunicationPeer
  • JavaDocs for packages and core modules (Bolek)
  • Remove underscore suffixes from attributes' names (?) (Bolek)
  • No console logs in unit tests (Bolek)
  • No constructor with source address for CommMessage (Bolek)
  • Updating objects on replicas (Bolek)
  • Better test coverage (40% ?) using JUnit and Mockito (Bolek + Grzesiek)
  • Better AppKey generation for new users (registration in central database + key generation?) (Grzesiek)
  • Stable bootstrap peer running on Naan. (Grzesiek)
  • Only EntryPoint and GenericConfiguration should use XMLConfiguration (Grzesiek)
  • Kademlia DHT instead of BDB (Grzesiek)
  • Cryptography from Gunnar's doc (Grzesiek)
  • Unified scripts that work on linux and mac (Grzesiek)
  • Clean TomP2P shutdown (Grzesiek)
  • Verification that only subclasses of CommMessage are sent/received through network (Grzesiek)
    • DHTMessage should be derived from CommMessage
  • Checkstyle-compliant Eclipse formatter config available on wiki (Grzesiek)
    • using source format in Eclipse should give no checkstyle errors
  • Filter module for synch/async messages (Piotr)
  • Transition to GIT (Piotr)
    • preserve change history (only for trunk?)
    • separate projects in separate repos? (i.e. "nebulostore", "nebulostore-systest", "nebulostore-bittorrent")
    • Gerrit running on Naan
  • Easy integration with nebulo-based projects (Rafał)
  • Move systest into separate project (and repo?) (Rafał)
  • Switch to Java 7 (Rafał)
    • make sure maven, naan, planetlab work
    • provide instructions on wiki how to make sure that eclipse is using java 7
    • talk briefly about java 7 features and where we can use them on one of our meetings
  • Easier GUI with optional help messages (Jadzia)
  • List tests (Jadzia)
  • Less trivial broker (Szymon)

Waiting list

  • list merging when replicas go up / down (Bolek)
  • thread pool in dispatcher - don't create a new thread for each job (Bolek)
  • lists with objects having repeated IDs: majority voting on what stays, what should be changed (check list merger, Bolek)
  • monitoring of my replicas/all peers & estimating resiliency (Szymon) (?????)
    • no optimization of storage contracts
    • "reasonable" replication rate (each replication group has 3 replicators?)
    • migrating data to new replicator (after a new contract is concluded, the broker sends update/asynch messages to the new replicator) (?)
  • cryptography for owner write authorization
  • permanent failures
    • detecting
    • planning the reconstruction (which data, from whom) (Piotr)
    • reconstruction
  • our own serialization / our own communication protocol
    • enable non-java clients
  • updates by diffs, not by complete files / lists (doesnt make sense)
  • subscription for object notifications (PeerSoN requirements) (done)
  • erasure codes instead of full replication
  • broker
    • external brokers: delegating availability maintenance to other peers
    • estimating resiliency
    • monitoring of my replicas
    • monitoring of all peers
    • peer search (find a peer with availability > 0.85)
    • replication contracts - decision (Szymon)
  • garbage collection (? maybe contracts are sufficient ?)
    • list elements
    • files
Last modified 5 years ago Last modified on 10/14/13 19:58:43

Attachments (1)

Download all attachments as: .zip