wiki:KrakenConfDB
Last modified 6 months ago

Kraken ConfDB

  1. Kraken ConfDB
    1. Motivation
    2. Features
    3. Getting Started
      1. Basic Operations
        1. Add
        2. Search
        3. Update
        4. Remove
      2. Transaction
      3. Flashback
      4. Rollback
      5. Object-Collection mapping
    4. API
    5. See Also
    6. History

Motivation

In the past, I used RDBMS and JPA for policy and configuration management. It was very painful, because relation schema was too restricted. In general, security solution's configuration schema is very complex. (nested, and nested detail options) You should normalize schema and create so many entity classes. Medium-sized project has hundreds of tables for configuration on average caused by normalization. How about to use a file instead? XML or JSON file will be better approach, however, they are not safe for power failure, concurrent access and modification.

On the other hands, Hibernate JPA is not OSGi friendly. You can use Kraken JPA (Hibernate backend) for database connectivity, but it's very troublesome on development phase. Because it uses DynamicImport-Package, and drains permgen space. In addition, standalone database is not appropriate for embedded solution.

If something has only pros of two approaches, RDBMS's concurrency control and File's flexible schema, development will become very easy. Kraken ConfDB is inspired by mercurial and couchdb. Since it has append-only file scheme, multiple readers can access same data without lock, and durable for power failure. Since it has document-oriented approach (schemaless), you can persist object graph easily. Since it has version control, you can change multiple documents atomically, and rollback to specific revision.

Features

  • Revision Control and Rollback
  • Conflict Detection
  • Object-oriented Database (schemaless)
  • Object-Collection Mapping (similar to ORM)
  • Concurrency control and Lockless reading
  • Lightweight and embeddable file database

Getting Started

Basic Operations

Add

Let's start with simple example code. There is an Animal class: 

public class Animal {
	private String type;
	private String name;

	// nullary constructor is required for instanciation
	public Animal() {
	}

	public Animal(String type, String name) {
		this.type = type;
		this.name = name;
	}

	public String getType() {
		return type;
	}

	public void setType(String type) {
		this.type = type;
	}

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	@Override
	public String toString() {
		return "type=" + type + ", name=" + name;
	}
}

Add some animals to config database: 

// create or open config database 'testdb' 
FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");

// add bombom
Animal cat1 = new Animal("cat", "bombom");

db.add(cat1);

// retrieve all animals in persistent storage
ConfigIterator it = db.findAll(Animal.class);
for (Animal animal : it.getDocuments(Animal.class)) {
	System.out.println(animal);
}

it.close();

You will see following output: 

type=cat, name=bombom

Above example code will create testdb directory. It contains changeset, manifest, and collection files. Since kraken-confdb is version controlled database, you can list all commit logs: 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");

for (CommitLog log : db.getCommitLogs())
	System.out.println(log);

This code will print all commit logs like this: 

rev=1, committer=null, msg=null, changeset=[op=CreateCol, col=org.krakenapps.confdb.file.Animal, doc=0]
rev=2, committer=null, msg=null, changeset=[op=CreateDoc, col=org.krakenapps.confdb.file.Animal, doc=1]

However, these commit logs don't contain any commiter or messages. Let's add new config document with commit log: 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");
Animal cat2 = new Animal("cat", "jiji");
db.add(cat2, "periphery", "added my cat");

for (CommitLog log : db.getCommitLogs())
	System.out.println(log);

output:
rev=1, committer=null, msg=null, changeset=[op=CreateCol, col=org.krakenapps.confdb.file.Animal, doc=0]
rev=2, committer=null, msg=null, changeset=[op=CreateDoc, col=org.krakenapps.confdb.file.Animal, doc=1]
rev=3, committer=periphery, msg=added my cat, changeset=[op=CreateDoc, col=org.krakenapps.confdb.file.Animal, doc=2]

Until now, kraken-confdb returned all config documents using db.findAll(). When you want to find specific config documents, use find(Class<?>, Predicate) or findOne(Class<?>, Predicate) methods: 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");
Config c = db.findOne(Animal.class, Predicates.field("name", "jiji"));
Animal jiji = c.getDocument(Animal.class);
System.out.println(jiji);

output:
type=cat, name=jiji

There are also other predicates, for example: 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");
ConfigIterator it = db.find(Animal.class,
	Predicates.or(Predicates.field("name", "jiji"), Predicates.field("name", "bombom")));

while (it.hasNext()) {
	Config c = it.next();
	Animal animal = c.getDocument(Animal.class);
	System.out.println(animal);
}

it.close();

output:
type=cat, name=bombom
type=cat, name=jiji

Update

Periphery has another cat named 'bandi'. She tries to add bandi: 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");
Animal cat3 = new Animal("cat", "band");
db.add(cat3, "periphery", "added bandi");

Config c = db.findOne(Animal.class, Predicates.field("name", "bandi"));
System.out.println(c.getDocument(Animal.class));

But this code does not work and throws NullPointerException. What's the problem? There's a typo. Let's update it: 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");
Animal fixed = new Animal("cat", "bandi");

Config old = db.findOne(Animal.class, Predicates.field("name", "band"));
old.setDocument(fixed);
db.update(old, fixed, false, "periphery", "fixed typo");

Config c = db.findOne(Animal.class, Predicates.field("name", "bandi"));
Animal bandi = c.getDocument(Animal.class);
System.out.println(bandi);

output:
type=cat, name=bandi

Find old document, change document, and call update(). Now, the config document (bandi) has two revisions. The third parameter of update() is ignoreConflict parameter. If another one already modified same document, confdb throws conflict exception by default. If you want to force overwrite other one's modification, set ignoreConflict parameter to true.

Remove

After one month, bombom is returned to home: 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");
Config bombom = db.findOne(Animal.class, Predicates.field("name", "bombom"));
db.remove(bombom, false, "xeraph", "return to home");

The third parameter of remove() is also ignoreConflict. If another one already changed or removed same config document, confdb throws conflict exception by default.

Transaction

You can use explicit transaction instead of implicit transaction. In other words, you can add, update, or remove multiple config documents atomically. 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");

ConfigTransaction xact = db.beginTransaction();
db.add(xact, new Animal("squirrel", "porori"));
db.add(xact, new Animal("sea otter", "bonobono"));
xact.commit("xeraph", "added more animals");

ConfigTransaction.commit() will commit all changes atomically. Of course, you can rollback transaction using ConfigTransaction.rollback().

Flashback

If you ran all examples step-by-step, current commit status should be: 

rev=1, committer=null, msg=null, changeset=[op=CreateCol, col=org.krakenapps.confdb.file.Animal, doc=0]
rev=2, committer=null, msg=null, changeset=[op=CreateDoc, col=org.krakenapps.confdb.file.Animal, doc=1]
rev=3, committer=periphery, msg=added my cat, changeset=[op=CreateDoc, col=org.krakenapps.confdb.file.Animal, doc=2]
rev=4, committer=periphery, msg=add bandi, changeset=[op=CreateDoc, col=org.krakenapps.confdb.file.Animal, doc=3]
rev=5, committer=periphery, msg=fixed typo, changeset=[op=UpdateDoc, col=org.krakenapps.confdb.file.Animal, doc=3]
rev=6, committer=xeraph, msg=return home, changeset=[op=DeleteDoc, col=org.krakenapps.confdb.file.Animal, doc=1]
rev=7, committer=xeraph, msg=added more animals, changeset=[op=CreateDoc, col=org.krakenapps.confdb.file.Animal, doc=6, op=CreateDoc, col=org.krakenapps.confdb.file.Animal, doc=7]

If you want to see config documents at a specific revision, say revision 5 (I want to see bombom!), use flashback mode: 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb", 5);

Config c = db.findOne(Animal.class, Predicates.field("name", "bombom"));
System.out.println(c);

output:
id=1, rev=1, prev=0, doc={name=bombom, type=cat}

In flashback mode, you cannot change anything. Any modification will cause exception.

Rollback

If you want to rollback changes to specific revision: 

FileConfigDatabase db = new FileConfigDatabase(new File("."), "testdb");
db.rollback(5, "xeraph", "rewind 1 month back");
Config c = db.findOne(Animal.class, Predicates.field("name", "bombom"));
System.out.println(c);

It creates new changeset with specific revision. You can change old documents after rollback. (not allowed in flashback mode)

Object-Collection mapping

API

See Also

History