root/04/release-0.4e/doc/kombilo.doc

This is the documentation for Kombilo 0.4 - a go database program.
It was written by Ulrich Goertz (u@g0ertz.de).
Kombilo is published under the GNU General PublicLicense (see the 
file 'gpl.txt' or look at http://www.gnu.org/copyleft/gpl.html).
This program comes WITHOUT ANY WARRANTY.

Most of the things explained here (and some additional material, including
pictures) can also be found in the tutorial, and in a more detailed and 
much nicer form. If you have already installed Kombilo, you should be 
able to access it via the Help menu. Otherwise, it can be found on the 
Kombilo website (see http://www.g0ertz.de/kombilo/tutorial.html).

----------------------------------------------------------------------

Contents:
  Getting started
  The menus
  The board
  The search window
  The SGF viewer
  Requirements on SGF files
  -----
  History
  To Do

---------- GETTING STARTED ------

I assume that you have successfully installed Python and that you can 
start the program. The main window with the board, and the search 
window will pop up.

First, you have to install a database where to search in. A database 
is simply a directory with SGF files in it. Open the 'Edit DB list' 
window in the File menu. Then add one or more directories. You can 
also add all subdirectories of some directory (and all of their 
subdirectories ...) simultaneously, by using the 'recursively add 
subdir's' option (this may take some time, though).

The games from the databases you added will show up in the game list 
in the search window.

---- Note: In some Python versions the widget for selecting a directory
is not available. (Technically speaking, they come with an old Tk version;
Tk is the programming language underlying the GUI toolkit.) As a
consequence, the program will crash once you press the 'Add DB' button
in the 'Edit DB list' window. I think that this is mainly a problem on 
Windows95/98 systems, but it may happen on other systems, too. 
There is the following work-around: Check the 'Old Tk version'-option
in the 'Advanced' submenu of the options menu. Now the 'Add DB' button
will bring up a widget to select a file, which is always available.
In order to select a certain directory which should be added as a
database, you have to select any file in it. 
It is still possible to add all subdirectories of some directory
at the same time, by the 'Add subdir's recursively' option. To do so,
you have to select a file in the main directory. (If it doesn't
contain any files, but only the subdirectories you want to add,
you have to copy any file to that directory first. Don't take
an sgf file because (probably) you don't want that sgf file to
show up in the database.) ------------------------------------------

Now you can just enter some pattern on the board, select a 
search-relevant region (with the right mouse key), and start the 
search in the search window.

You might also want to look at Kombilo's web site with 
screenshots and a description of many of it's features at
http://www.g0ertz.de/kombilo/

---------- THE MENUS ------------

­-- File

Open: Load an SGF game. You can look at the game on the main board. 
      This is useful to analyze (your own) games, or maybe to look 
      at a joseki library. (In principle, this works well with
      Kogo's joseki dictionary, e.g., but unfortunately it takes
      very long to load the dictionary, since it is such a large file, 
      and the SGF Parser, which is written in Python, is not very fast.)

Edit DB list:
      With this menu item, you open a window where you can edit which
      databases (i.e. games from which directories) you want to work with 
      for the next search. 

      A database corresponds to a directory; it contains all the SGF 
      files in that directory. Files in subdirectories do not belong 
      to the database (but it is possible to add all subdirectories
      as databases of their own at the same time: use the "recursively 
      add subdir's option).

      See also the section "Requirements on SGF files" below in order 
      to understand which kind of SGF files the program can handle; 
      basically, it handles simple game records, they may contain
      variations, but the variations are ignored for the search.
      All filenames have to end in '.sgf'.

      In the Edit DB List window, you can add databases and remove them. 
      If you add a directoy for the first time, the SGF files have to be 
      'processed', i.e. the SGF files are translated into a format that 
      is suitable for the search function. This information is written to 
      several files with the extension .db. The processing takes quite
      some time, but has to be done only once (unless you change the files
      or add/remove files from that directory). Afterwards, the SGF files 
      are not needed for the search itself anymore. But of course they are 
      needed when you want to look at a game, so they should be left where
      they are.

      You also can reprocess a directory, in case you added or deleted SGF 
      files in that directory.

Export search results:
      This opens a small text editor with some information on the previous
      search, i.e. the search pattern; the search pattern with continuations;
      some statistical information. You can edit this information (but the
      editing capabilities are pretty limited), and save the result to a file.
      This might be useful to store information on some search for yourself,
      or to send it to someone via email, or to post it to a newsgroup
      like rec.games.go. If you want to write a page for Sensei's library,
      choose the 'Wiki' export style in the Options menu.

--- Options

  Edit custom menus
      The custom menus can be used to add your own menu entries.
      Upon selecting a menu entry, Kombilo can do a pattern search for some
      pre-defined pattern and/or a game info search and/or open a html file in
      your web browser. For example, you could create entries for fuseki or
      joseki patterns, for players, or for titles.

      To edit the custom menus, select the corresponding entry in the Options
      menu. You see a list of the currently existing menus, submenus and
      entries. The first line with a * represents the Kombilo main menu. You can
      add submenus or entries to the menus, or delete them.

      When an entry is selected, you can  


      * Add pattern information by pressing the corresponding button. The pattern
        (and search-relevant region, and the search options) will then be
        associated with this menu entry).

      * Add game info information by clicking the corresponding button. A window
        will pop up where you can enter the information to search for.

      * Add a HTML file by entering the file name in the corresponding field, or
        by browsing for a file.

  Fuzzy stone placement
      Place the stones on the main board slightly off the exact
      point, in a random direction, to make the position
      look more natural. (Well, some people might think that it
      is just ugly, so you can switch it off here).

  Shaded stone mouse pointer
      (Don't) Show the current position of the mouse pointer on 
      the board and the color of the next stone to be played
      by a shaded stone.

  Show next move
      In case a SGF file has been loaded, show the position of the
      next move with a circle.

  Jump to match
      This controls the behaviour of the SGF viewer when you open
      a game from the game lis tafter a pattern search. If this 
      option is checked, the viewer will jump directly to the position
      where the pattern you searched for was found in that game.

  Smart fixed color
      If this option is enabled, the 'fixed color' option will be 
      automatically enabled when you select the whole board as 
      search-relevant region, and disabled when you select a smaller 
      region. (You can nevertheless change that after selecting the 
      region and before starting the search.)
      This is useful because if 'fixed color' is not used, Kombilo 
      regards a position and the same position with swapped colors as 
      equivalent; in the case of whole board searches that can lead to 
      counter-intuitive results when you look at the continuations 
      (e.g. place a black resp. white stone on the upper left resp. upper 
      right hoshi, do a whole board search without 'fixed color', and 
      look at the continuations).

  Export style
      This option controls the format in which information on the 
      previous search is exported (compare 'Export search results' in the
      File menu). You can choose between ASCII-style, which is just plain 
      text, and is the standard. E.g. it is suitable if you want to post 
      information on some search to the usenet newsgroup rec.games.go. The 
      other choice is Wiki-style which is useful to write pages for 
      Sensei's Library (http://senseis.xmp.net/).

  Uppercase labels
      If you want to use the 'Export search results' function to
      produce output for Sensei's Library, it is useful to use
      lowercase labels for the continuations, since only lowercase
      letters are automatically understood by Sensei's Library. 
      If you do not want to do that, and find that uppercase
      labels look better, you can use this option.<p>

  Show color swap
      If this is active, hits for the pattern with reversed colors will
      have a minus sign. 

  Save options
      Save the options; this also saves the current size and position
      of the two main windows.

  Advanced Options:

      Number of remembered searches
          As we have seen, with the 'back' button you can jump back to 
          the previous search. This option controls the number of previous 
          searches that are remembered. The default is 5, and if your 
          machine has only a small amount of memory (less them 64MB, say), 
          you probably should not set it much higher, or Kombilo might run 
          out of memory and crash. 
          On the other hand, if you have lots of memory, it might be 
          convenient to set it to a higher number, or even to 0, which 
          means 'no limit': all searches are remembered, as long as there 
          is enough memory.


      Alternative SGF viewer (Unix/Windows only):
          If you want to use your customary SGF viewer/editor instead of
          the viewer coming with Kombilo, enter the command to start it
          and the command line options that tell it to open a certain
          sgf file here. For example, on my Linux machine, I could use

          /home/ug/cgoban-1.9.11/cgoban
          -edit %f

          in order to use CGoban. In order to use WinMGT under Windows,
          enter something like

          c:\winmgt\winmgt
          %f

          Of course, in both cases you may have to adjust the path,
          and possibly the command itself.
  
          Delete the command in order to use v.py.

      Sloppy SGF parsing
          If this is enabled, Kombilo will accept SGF files which
          have multiple occurrences of the same tag in a node, or
          which have line breaks in move tags. Both of these are
          forbidden by the SGF standard.

      Use C search routine
          Enable or disable the (much faster) C search routine. If the
          C search function could not be imported, this option is 
          disabled. See the file install.txt for information on
          how to install the C search function.

          If you think the C search routine is still too slow, just
          use the Python search a couple of times. When you switch 
          back again, you will be enthusiastic about the C search :-)
          But seriously: I do intend to improve the search engine 
          further, but I think that currently there are more important
          things to do.

      Duplicates
          This option says what the program should do with games that
          are already in the database, but are about to be inserted again
          (by 'AddDB'). You can either completely omit the duplicate check
          (to save some time during the processing), always accept (but get
          a warning message) or always reject duplicates, or have the 
          program ask in each single case.
          If you use the 'strict duplicate check' option, Kombilo will
          not only compare the symmetrized Dyer signatures of the games,
          but also the final positions. This makes is much less likely
          that different games are shown as duplicates. On the other hand,
          if the same game occurs twice, but with a different orientation,
          or with minor errors in the game record, the strict duplicate
          check will not find it. Thus you probably should use this option
          only in special cases.

      Old Tk version
          On (some?) Windows9x systems, Python doesn't come with a 
          widget to select a directory. If this is the case for your 
          system, and the 'Old Tk version' option is not checked, the 
          program will crash when you press the 'AddDB' button. If you 
          enable this option, a widget to select a file will be used 
          instead.

      Customize appearance
          On choosing this item, you get a window where you can fine tune
          several fonts, font sizes, and colors that Kombilo uses. You can
          also change some global GUI options by editing the file 'kombilo.app'.


--- Help

  Information about the program, this documentation,
  and the license. If you choose 'Tutorial', the Kombilo tutorial will be
  opened in your default web browser.

­--------- THE BOARD ------------

--- The board itself:

  Place stones:
    Just click on a point in order to place a stone.

  Remove Stones:
    Press the control key, and click on a stone in order to 
    remove it.

  Wildcards:
    In order to include wildcards in your search pattern, i.e.
    points which may be empty or contain a stone of either color,
    press the shift key and click on an (empty) point. Wildcards
    are shown as small green circles.

  Selection: 
    With the right mouse key, you can create the rectangle that 
    is relevant for the next search. (Click on the top left point,
    and drag the bottom right point to where you want it). Everything 
    outside this rectangle is neglected. If no rectangle is shown,
    the whole board is relevant.

    If the region contains a corner, it will be mirrored and rotated
    for the search, but not translated. If it contains some points on
    the edge, but no corner, it will be mirrored, rotated and translated
    along the edge. Otherwise, it will be mirrored, rotated and translated,
    such that the position may occur anywhere on the board (not right in
    a corner, nor right at the edge, though). (But see the
    'Fixed Anchor' option in the search window.)

  Automatic search:
    Double clicking with the left mouse key is the 
    same as first clicking once, then starting a search. This is 
    particularly convenient in order to go through a 'variation tree', 
    for some joseki, say. See 'show continuations' below.

--- Navigation:

  If an SGF file has been loaded, you can navigate as in the SGF viewer 
  (see below).

  Otherwise, the "next button" and "end of game button" do nothing, the 
  "previous move button" takes back a move, the "start of game button"
  clears the board.


--- Show continuations ('ABC' button)

  This lets you display the continuations from the found games in the pattern 
  you searched for. Only continuations in the current search-relevant region 
  are shown.

  If only black/white moves occured at a certain point, the corresponding 
  letter is written in black/white. Otherwise it is written in gray.
  (This color encoding does not work for continuations at non-empty 
  intersections, i.e. "under the stones". Furthermore, the color encoding
  gets lost when you resize the board; click 'Show continuations' twice
  in order to restore it.)

  During the search, the continuations are recorded anyway. This button just 
  displays them or removes them from the board.

  If the continuations are displayed, you can easily "walk through the 
  variation tree" by double clicking on the corresponding points on 
  the board, because double clicking on the board is (always) the same 
  as a simple click (=placing a stone) plus performing a search.


--- Color of next stone:

  With the four buttons left to the navigation buttons, you can choose if 
  the stones you put on the board should be
  - alternating black/white,
  - alternating white/black,
  - all black
  - all white.

  The shaded stone which appears while the mouse pointer is on the board 
  always shows the color of the next move.

---------- THE SEARCH WINDOW ----

--- The game list

  All games are shown as 
    filename(without .sgf): white player - black player (winner)

  The game list shows all the games that are in the current databases 
  and that contain all patterns from the previous searches. In other words, 
  when you start the program, all games from the databases in the current 
  database list are there. If you search a pattern, the new list contains all 
  games in which that pattern occurs. This new list will be the basis for the 
  next search, unless you 'reset' the list first.

  Thus it is easy to combine searches (e.g. games by Cho Chikun in which the  
  nadare joseki occurs): Just search for one criterion first, then search for 
  the second one without resetting the game list. 

  After a search, the list of results is appended for each game. '123D' means 
  that the pattern occurs in move 123, and that the continuation is D. '245' 
  means that the pattern occurs in move 245, and that no more move is played 
  in that region.

  The number of games that are currently in the list as well as the
  black/white winning percentages are shown on the top.

--- Statistics

  The statistics window in the lower right shows the ten most common 
  continuations, and how often they have been played. Furthermore,
  it shows the number of matches, the number of matches with colors
  as on the board (resp. reversed), and the B/W winning percentage.

--- Back button

  Go back to the previous search: this restores the board and the
  game list.

--- Reset button

  Reset the game list, i.e. put all the games into the game list that are 
  in the databases of the current DB list.

--- Fixed Anchor

  Restrict the search to match only games where the given pattern occurs at 
  the exact position. For example, if you enter a pattern (and selection of 
  relevant search region) like this
 
    ...
    .O.
    ...    , the O stone being on the 5-5 point

  the usual search will find all games where a stone with the 8 adjacent 
  points being empty occurs, anywhere on the board (except for in the corner 
  and at the edge).

  If you use the fixed anchor option, only games will be found where the 
  stone is on the 5-5 or 5-15, 15-5, 15-15 point. I.e., the only way in 
  which the found pattern might differ from the original one is by rotation/
  mirroring of the whole board.

--- Fixed Color

  Only finds games where the pattern occurs with the same colors as it was 
  entered. If it is not used, a color reversal is allowed. It is highly 
  recommended to use this option when doing a whole board search (for some 
  fuseki lines, say), because otherwise the continuations may look 'funny'.  
  (Search for the empty board without 'fixed color' in order to see what I 
  mean.)

--- Move Limit

  Only finds games where the pattern occurs before the given move. If set 
  to 250, there is no move limit, i.e. the whole game is relevant.

  If a move limit is set, it also applies to the continuations. If the first 
  continuation in some pattern occurs after the given limit, this is treated 
  as if there were no continuation.

  Usually it doesn't make the search much faster to set a move limit. It just 
  might be helpful in order to get more useful results.

--- Game Info Search
  The game info search allows you to search for games (among those
  that are currently in the game list) played by certain 
  players, in a certain event, in a certain time period etc. 
  You can also do a general text search in the SGF file (with the 
  'Anywhere' entry). In this way, you could search for games won by
  white (search for 'RE[W'), for games played under chinese rules
  (search for 'RU[Chinese]') etc.

  If you enter something in several fields at the same time, only games
  will be found which match all these entries.
  

--- Signature search
  Here you can search for games with a certain Dyer signature.
  The signature of a game is given by the coordinates (in SGF 
  format) of the moves 20, 40, 60, 31, 51, 71. This almost 
  always characterizes a game uniquely.

  If you press the 'signature search' button, and a window will
  pop up, where you can enter the coordinates of the corresponding
  moves. If you click on an intersection on the board,
  the corresponding coordinates will be entered in the
  currently active text entry below, and the next entry will be made 
  active. So you can enter the signature simply by clicking on
  the places where moves 20, 40, ... were played. You can also omit
  some of them (in most cases, two or three of the moves will
  be enough to characterize a game uniquely).

--- Pattern Search
  This starts the search of the pattern on the main board, among the games that 
  are currently in the game list.

---------- THE VIEWER -----------

The menu commands should be self-explanatory; compare the explanations above.

After loading an SGF file, you can navigate through it in the following way:

Click on the board: 
  By clicking on the point where the next move  is made, the stone will be 
  placed there.

  This is of course most useful if the option "show next move" in the options 
  menu is enabled; in that case the position(s) of the next move are marked 
  with circles.

  This method is particularly convenient (I think) if there are variations. 
  Unfortunately, this is currently the only way to choose among variations. 
  So if your SGF file has variations, you have to enable the "show next move" 
  option (unless all variations are marked with labels in the SGF file).

  If the next move is a pass (or a pass occurs in one of the variations),
  the pass button is active.

  On the other hand, if you disable the "show next move" option, you can use 
  the viewer as a very simple program to let you guess the next move.

  If you click on a point for which there is no move in the SGF file, nothing
  happens. In the current version it is not possible to enter your own 
  variations.

Next move button (or left-arrow key):
  Goes to the next move. If there are variations, the first variation is 
  chosen.

Previous move button (or right-arrow key):
  Goes back one move.

Fast forward button:
  = 10 times "next move"

"Fast rewind" button:
  = 10 times "previous move"

Start of game button, End of game button:
  Go to the start/end of the game.

Note: You can also use the viewer separately from Kombilo, in
order to look at SGF files. Just start it with
'python v.py', or 'python v.py filename', in order to look
at the SGF file filename.


------ REQUIREMENTS ON SGF FILES --------

There are a few requirements on the SGF files that are used in the 
databases. They will be satisfied by ordinary game records, but 
might not be satisfied by "strange" SGF files.

First of all, the filename of an SGF file always has to end in '.sgf'.

The program does not handle several game records in a single SGF file. 
All but the first record are simply ignored.

The program expects that in the game record there are alternating black 
and white plays. Variations are ignored; only the first (main) variation is
followed.

In addition, at the very beginning an initial position can be set up. This 
is what happens in handicap games, for example. So handicap stones are treated
correctly. It is also possible to set up an initial position consisting
of black and white stones, like a go problem. On the other hand, "during the 
game", i.e. after the first black or white move has been played, no stones may 
be added or removed except for the ordinary alternating black/white moves (and 
except for captures, of course).

All moves after a pass are ignored.

The viewer does accept most SGF features, I think. In particular it handles 
variations (the navigation has to be done by clicking on the concerning
points on the board), and adding/removal of stones during the game. It 
displays labels, but it does not properly display text labels with more 
than one letter/digit.

It ignores some of the new SGF tags like "good for black", "bad for white", 
... 

======================================================

History:

I had been loosely thinking about writing a go database program 
like Kombilo for quite some time, but I didn't start to write some code 
until

June 2001:   Started experimenting with a simple pattern search function. 
             Since it was easier than I first thought to produce a basic 
             search engine I decided to continue with this project.

Nov. 2001:   Version 0.1 released.

Jan. 2002:   Version 0.2 
             with an improved user interface, duplicate check, winning
             percentages, more comfortable game info search, a workaround
             for the 'choose directory' widget which is missing in some
             Python distributions, ...

April 2002:  Version 0.3
             with an improved search engine, hash tables for joseki and
             fuseki search, winning percentages for the continuations,
             faster SGF parser, export function for search results, ...

May 2002:    Version 0.4 (testing)
             custom menus; better handling of large databases; SGF parser
             rewritten from scratch (partially in C++); edit comments,
             game info of SGF files.

======================================================

To Do:

Here are some ideas on features that I might implement in one of the next 
versions. If you miss anything that is not on the list yet, please
drop me a line!

- Include a full SGF editor instead of the SGF viewer.

- Make the search function consider variations.

- More database administration functions: save/load/
  merge/manually edit game lists ...

- Implement some kind of fuzzy search (?)