Anji - The Opensource Multivalue RAD environment


What is Anji

TODO: Spellcheck whole document Anji is a web-based IDE for multivalue data (a little like bespin). Anji is designed to allow people to build applications quickly. It is based upon some old principles which have become unfashionable whilst the IT industry has dashed headlong into Windows GUI and web based applications. In contrast Anji allows an ordinary Business user to build their own application. In many respects, It's goals are fairly similar to Ruby-on-Rails, although the implementation is entirely different. Inspiration for Anji has come from many places such as Advanced Revelation, and more recently, Bespin (http://labs.mozilla.com/2009/02/introducing-bespin/)
In short, Anji is the tool for the multivalue developer in the modern world. You can read it's history here Welcome. This is an a Opensource of Anji from Billabong Services ltd. As such support is limited Currently it is only supported under Ubuntu 10.04LTS with OpenQM version 2.6-6
It will probably run under other environments.
It allows for the developer to build and run multivalue applications entirely through a web browser.
There is a change log here , but this is not fully maintained.

Licence


Anji is licenced under the GPL version 2. If you need any other form of licence, please contact us, and well try to accomodate you. Commercial licences are available for 50GBP per server. External contributors will be eligble for free commercial licences. Portions copyright Glen Batchelor, Tom DeLombarde, Ladybridge Systems and possibly others.

Contributions

TODO: Define licence for contributions. These are welcome, but must be provided under the LGPL version 2 or the Apache 2.0 licence.

Requirements

Anji has relativly low hardware requirements. We reccomend 500Mhz Pentium II, 256MbRAM, 4Gb disk, CDROM, and a network card. You can probably get away with much less if you need to. NB. Firefox is the reccomended browser. Microsoft IE is not supported (see below).

Microsoft Internet Explorer

The choice of browser support for Anji has been a difficult decision to make. One the one hand, we wish for the broadest browser support possible. On the other hand, it has become clear that this will take a huge amount of resource to achieve. This is far from ideal! Removing Microsoft IE from the list of supported browsers resolves a lot of issues, therefore it is unsupported, until such time as it improves. Having said that; there is nothing that specifically excludes IE.

Since I wrote the above, I have discovered another major flaw of IE. Within forms, it sends in the GET/POST request the text on the button, not the value defined in the html. This is completely wrong, and it destroys any attempt to build Anji with international capabilities. I can't put different languages on the buttons without coding around it. Look at http://www.fourmilab.ch/fourmilog/archives/2007-03/000824.html for more info

Lest there be any doubt, Anji does not support Microsoft Internet Explorer. I do this with a heavy heart, as I know this will cause some pain in deployment - after all, IE on Windows is currently the majority platform. As soon as the IE issues are resolved, then Anji will support IE. Either Microsoft fixes it, or we code around it. For an independent review of how much IE is holding back the web, look here

Local time zones

Historically, multivalue systems have stored the local time in database time fields. This was just about usable if all users were attached to a single computer, but started to look inadequate whne multiple timezones are encountered. Considering the USA origins of multivalue, where there are multiple timezones this is completely inadequate. Twice a year daylight-saving timezone issues will be encountered, which can lead to database ambiguities. In the modern world where users from all across the globe are encountered, the use of a arbitary local time is completely inadequate. For this reason Anji uses UTC internally, and converts to the whatever timezone the user sets in their preferences.

Display format

Multivalue gives you a lot of choice on how dates are displayed. In practice, this is more of a curse than a blessing. Particularly now that the Internet has made most systems globaly accessible. For this reason, there is now a default date format of YYYY-MM-DD, which is the ISO 8601 format. The general principle that Anji implements is one of storing data in the most widely recognised unambiguous international format - a neutral point of view. End-user display MAY optionally be localised, but that is the responsibility of the user interface designer. Anji does provide some facilities here, but they are incomplete and on the TODO list.

Datatypes

multivalue systems are loosely datatyped. The datatype is hazily defined by the dictionary conversion, and is not enforced in any way when a record is written. this gives great freedom and flexibility, but comes at the cost of data consistency. Once you have a loarge database, you start to curse the little bits' of dirt that creep into the dataset. For this reason, Anji has some fixed datatypes, which are enforced during the write of a record (transactional write). The datatypes :- Boolean, Datetime, String, Currency, Integer. More will be added, and defined on a separate page as soon as I get a chance to write it up.

Dictionary types

Anji supports the OpenQM dictionary types of C, D and I. The Pick style A and S types are not supported, but will work to some extent. If you have these types in your account (possibly from a migration), then it's a good idea to create new items, with different names. It's a mess, but that's what happens when systems have a long history.

Upper and Lower Case

Anji handles case mapping differently from the traditional multivalue way, which emerged from the days of upper case only terminals. Anji stores ALL strings as case dependent, and so "McGregor" is different from "Mcgregor". If you need to search on a field, this can be problematic, so create a symbolic field which holds the data in all UPPER case, and do an upper case search on that. See here for details of how OpenQM handles case internals.

Installation Instructions

Here follows the complete instructions that we used to make a redundant Compaq Proliant 1850R into an Anji box. We chose this because we had a few lying around doing nothing, and because they have a decent hardware based hot-swap raid system. We don't like to run any server without RAID. Even an experimental one - life's just too short.
Your requirements will quite likely be different; so adapt them to suit yourself.
We only install the bare minimum for Anji to function. If you require other things like PHP, then add them yourself.

Load up Linux

We used Ubuntu 10.04 Server It was very simple. We chose the LTS version of Ubuntu so that these instructions can remain stable for longer. Download the CD, and get cracking. You can boot with the "custom" option to reduce the install size, alhough we did not do this. We took the default options for everything, except the following:- You can also use the 64bit Ubuntu, but you might need to make some changes. You will have to compile OpenQM in 32bit mode (notes needed), and also add the ia32-lib package. Take a look here

Add a swapfile

I prefer to use a swapfile, instead of a deadicated swap partition. I've chosen a size of 2048Mb (2Gbyte) in the root. Adapt as you like Verify using "free" command, which should show that you have swap memory

Change network card

Just in case you ever need to change the net card, here's how to do it:-

Load Apache

Load ScarletDME

This can be found on the downloads page, or you can use subversion to get it. 
apt-get install subversion
cd /usr
svn export https://scarletdme.org/svn/qmvdb/openqm/trunk/ qmsys
If you need to, you can also download the source code for revision 93here
Or you can follow these code snippets:- 
cd /usr
wget http://billabong-services.co.uk/anji/scarlet_R93.bz2
tar -xvjf scarlet_R93.bz2
cd qmsys
touch /usr/qmsys/errlog

Patch the QM k_error.c bug

Detailed here. This is required by Ubuntu 11.10 (and others) 

./buildgpl
 This step wil likely not work! Edit the buildgpl script, and remove -Werror from the QM_GCC_OPTIONS line, and try again. If that don't work, try adding -Wno-packed. If that don't work , try adding -Wno-pointer-sign. If you are using Redhat, Fedora or similar, you will need to remove the -Werror, and insert -Wno-pointer-sign.

Load OpenQM

This is an alternative to Scarlet (see above), and is infact just an outdated release of the code which Scarlet was based upon. It can be found on the Openqm.com site, which requires registration. Please do this so that Ladybridge are aware of your support and interest in their products. They don't bombard you with marketing junk, but you might get a annual newsletter. Nothing much else. If you need to, you can download the source code for version 2.6-6here
Or you can follow these code snippets:- 
cd /usr 
wget http://billabong-services.co.uk/anji/qm/qmsrc_2-6-6.tgz
tar -xvzf qmsrc_2-6-6.tgz
mv gpl.qmsys qmsys
cd qmsys
touch /usr/qmsys/errlog
./buildgpl
 This step wil likely not work! Edit the buildgpl script, and remove -Werror from the QM_GCC_OPTIONS line, and try again. If that don't work, try adding -Wno-packed. If that don't work , try adding -Wno-pointer-sign. If you are using Redhat, Fedora or similar, you will need to remove the -Werror, and insert -Wno-pointer-sign.
For Ubuntu 11.10, I needed the following:- 

QM_GCC_OPTIONS="-Werror -DLINUX -D_FILE_OFFSET_BITS=64 -DGPL -g"
QMLIBS="-Wl,--no-as-needed -lm -lcrypt -ldl"

Install qm startup files

Once you have got it to cleanly compile, you need to install the startup files. These have been constructed by Tom deLombarde of Blackflute system. If you're not installing a debian system, change the "cd" command appropriately. 
cd /tmp
   wget http://billabong-services.co.uk/anji/qm_etcinst.tgz
   tar -xvzf qm_etcinst.tgz
   cd etc
   cd debian
   ./install.sh
You will be asked if you want to enable qmclient and qmserver. Anji does not use this, so you can say "no" unless you have a goor reason to use the qm client/server services..

Patch the QM bugs (optional)

Most are small, but they are detailed here

Set OpenQM permissions

From the command line:- 
cd /usr/qmsys
   chmod -R o+w *
   chown -R www-data:www-data *

The unix permissions are a perennial problem. They are just not up to the modern day requirements of multiple user groups. In-house we are considering implementing Netware style trustees ACLs. We have tried the Posix ACLs and thay are even worse than normal unix permissions!
create a group for everybody who needs to run qm. 

   sudo /usr/sbin/groupadd qm
Add the web-server into the qm group:- 

   sudo /usr/sbin/usermod -a -G qm www-data
Change permissions on the qmsys files 

    cd /usr
    sudo chgrp -R qm qmsys
    sudo chmod -R 775 qmsys

Alternative Ubuntu instructions from Brian Speirs

An alternative description for Ubuntu can be found here

Redhat/Fedora installation notes from Gene Buckle

An alternative description for Redhat (and similar) can be found here (Cached)

Enable QM/SD error logging

edit /etc/qmconfig and place ERRLOG=10 somewhere in the middle. Doesn't matter where. Also set SORTMEM=16384 File should look like this 

[qm]
QMSYS=/usr/qmsys
ERRLOG=10
MUSTLOCK=0
NETFILES=0
NUMFILES=80
NUMLOCKS=100
NUMUSERS=40
OBJECTS=0
OBJMEM=0
SORTMEM=16384
SORTWORK=
TERMINFO=

Troubleshooting the QM install

You should now be able to telnet/console login as a user that is a member of the "qm" group. Enter into QM's TCL prompt by issuing the qm command. You will probably be asked "Current directory is not a valid account. Create account?" to which you should say no. If this does not happen, go back, and check the steps so far.

Install OpenQM documentation

From the command line:- 
cd /var/www
mkdir openqm
cd openqm
wget http://billabong-services.co.uk/anji/qm/qmhelp_2-6-6.zip
unzip qmhelp_2-6-6.zip
Test that this is available on the server simply by pointing your browser to the openqm directory.

Patch OpenQM 2.6-6 so it can index complex symbolic fields

This is documented here navigate to the OpenQM source file /usr/qmsys/gplsrc/dh_ak.c, and change 
   ak_header.itype_ptr = ak_header_size + DH_AK_NODE_SIZE;
to 
   if (header.file_version < 2)
    {
     ak_header.itype_ptr = DH_AK_NODE_SIZE + ak_header_size;
    }
   else
    {
     ak_header.itype_ptr = 2;
    }
Then recompile with the buildgpl script, as above

Download the Anji account

The preferred download is what I call "current". This is always held at http://billabong-services.co.uk/anji/anji.bz2. If you want to run the 11.05 stable release, then this is at http://billabong-services.co.uk/anji/anji_11_05.bz2 From the command line:- 
cd /home
wget http://billabong-services.co.uk/anji/anji.bz2
tar xvjf anji.bz2
chown -R www-data:www-data /home/anji
cd /home/anji
qm  (If you are asked to upgrade the VOC, reply "yes".)
BASIC BP INSTALL
RUN BP INSTALL

Install the CGI link

The cgi link is very simple script file. It looks like this:- 

#!/bin/sh
cd /home/anji
export TZ=UTC
cat - | qm -quiet "BGATEWAY"
Put these four lines of code into a file called /usr/lib/cgi-bin/anji. Make them executable like this:- 

chmod +x /usr/lib/cgi-bin/anji

Protect the CGI folder (optional)

As Anji is not fully secure, you might wish to use the built-in Apach htaccess security. This is a quick hack, and it would be far better to do this as part of a co-ordinated security strategy. Edit /etc/apache2/apache2.conf and put in this:- 

<Directory /usr/lib/cgi-bin>
  AuthName "permission"
  AuthType Basic
  AuthUserFile /etc/apache2/.htpasswd
  Require valid-user
</Directory>
Then add some users:- 

htpasswd -c /etc/apache2/.htpasswd username
Restart Apache with:- 
/etc/init.d/apache2 restart

Test this in your browser

Chrome, Firefox, Opera and Safari are supported ().
For example http://192.168.1.176/cgi-bin/anji (Substituting the address of your server for 192.168.1.176).
You should get a nice screen. If not, then it's time to troubleshoot.
Tryout BFILES, as it's a file browser. Click on a blue number to look at a data file or a dictionary. Click on an entry to edit it.
Take a look at BLIST, as it allows you to view a file - E.G. LIST BUSER FULLNAME PASSWORD
Try out the Spreadsheet export within BLIST (TSV option)

Install the full Webster dictionary

The AJAX demo programme which allows lookups of the Webster dictionary may not have all of the required contents. You can download the full contents like this:- 
 
cd /home/anji
wget http://billabong-services.co.uk/anji/WEBSTER.bz2
rm -r WEBSTER
tar -xvjf WEBSTER.bz2

Adding Anji to an existing account

This can be done by installing Anji, then from the welcome screen, select "BAccountInstall - Copies Anji into an already existing account"

Editing programmes

You can use the form editor, or your own as you wish. You can read notes here

Technical notes

www-data is where you might want to put html files for Apache to serve up (such as the QM manual). Anji itself makes no use of it. However, www-data is also the name of a the Apache user, so must be in the qm group, and also must have FULL read/write permissions to the any files it needs to write to. The simplest way to do this is to give www-data ownership to the anji directory and files. This is not good practice from an admin point of view, but the best hack I've been able to come up with.
It is vital that only one copy of QM exists on a system. Multiple copies will try to access the same shared memory area, and possibly lead to data corruption.

TODO

Links

This is a simple section to provide useful resources that might be of help. OpenQM_XML from Glen
Apt cache

Contact

Please contact us with feedback, bugs, contributions etc. Email is anji AT removethisbitbillabong-services.co.uk

Copyright Billabong Services Ltd. Last modified: 2012-02-15