26 January 2015

Chess Improvement Programs: Directed vs. Eclectic


Although people's learning styles vary significantly - meaning that there is no "one true path" to mastery that will apply to everyone for any skill - it seems obvious that possessing some type of chess improvement program is important for eventual success in gaining playing strength.  ("Do study techniques matter in chess?" previously addressed this idea in a broader sense, as did "Reflections on Training".)

Having a program - in other words, a structured approach to training that contains achievable goals - makes it more much more likely that you will in fact stay on your chosen path, rather than simply studying on a whim at random times.  "Whimsical" training can be done for fun and in some cases may make for a useful intellectual break from other tasks, but it is unlikely to produce significant results over time.

That leads to the question of how to construct a training program.  Here I would like to highlight some different types of resources available that are aimed at adult chess improvers, though they also could apply to up-and-coming juniors.  I break them down generally into "directed" programs, which are intended to be all-inclusive from a single source, and "eclectic" ones, which rely on independently gathering together chess resources.  Directed programs are by nature more highly structured (and demanding), which has its pluses and minuses.  I'll also discuss my own eclectic program as an example of that type.

Directed programs

A directed program, whether with a personal chess coach or following a set lesson plan and materials, is explicitly designed to achieve a set level of skill mastery.  We are all familiar with the "directed" educational programs involved in secondary education (typically through age 18), which aim (with varying results) to provide all students with a grasp of mathematics, science, literature, reading/writing, arts, etc.  The time commitment and need to follow a set plan week after week are what sets these programs apart - and make them more challenging to keep up with.
  • The most recent example of a full, amateur-to-master improvement course is that offered by Chess.com University through their "Prodigy Program".  Despite the name, it is open to adult students and the motto is "Master chess at any age, within 5 years".  Core features include: 7-page PDF weekly study plans; monthly online correspondence tournament; monthly master-level simul; and analysis of students' games.
  • The International Chess School's "Grandmaster Package" has been available for several years now, delivered online with PDF workbooks and video lessons; samples are available at the linked website.  An unofficial forum about the course was set up by former blogger Blue Devil Knight, although it is only active periodically.
  • Another example of a "comprehensive" packaged program is at GM Igor Smirnov's "Remote Chess Academy" which features his "Self-Taught Grandmaster" software along with some more specialized packages on openings and subjects like "How to Beat Titled Players".
  • Tiger Chess by GM Nigel Davies uses a subscription model, with currently 160 weekly lessons available, a monthly clinic with GM Davies, access to resources on openings, and guidance on outside books and internet resources.  A reduced membership level offers some introductory video courses.  A "Chess Improver" blog post talks more about the breakdown of time for various study topics.
  • The granddaddy of comprehensive courses, before online versions existed, was GM Lev Alburt's Comprehensive Chess Course.  Its series of instructional books is not fully integrated, but does run from beginner to advanced material and covers openings, tactics, middlegame strategy, and endings.
  • The modern version of beginner to master chess books is the three-set, nine-volume series by GM Artur Yusupov, under the set titles "Build Up Your Chess", "Boost Your Chess" and "Chess Evolution".  In this case, he breaks up the different topics (calculation, tactics, strategy, positional play, endgame, openings) by strength levels, so each of the three book sets is designed to be comprehensive for its particular level.
Eclectic programs

The line between "directed" and "eclectic" is not necessarily a hard one.  For example, if you follow Yusupov's book series in order, that is certainly structured learning, but not quite at the same level as receiving weekly lessons and exercises to complete.  Eclectic programs by nature rely on your interests and choices to drive the improvement process, which may be more fun and better match your own desires and personality.  The downside is that you will need to put more effort into determining what to study and in evaluating materials beforehand, rather than having it packaged up for you.

"Eclectic" does not necessarily mean disorganized.  Here are some resources designed to help the chess improver get organized for training:
  • IM Igor Khemeltsky's Chess Exam and Training Guide is a book for those looking to diagnose their strengths and weaknesses and receive follow-up guidance, this may be a good place to start.  He also offers an exam book on tactics.
  • Chess.com has "study plans" available on the site which link in an organized way to articles and videos available there.  Chess Mentor courses contain everything from basic tactics to advanced endgames, while the Study Plan Directory provides comprehensive links for training from beginners to intermediate (Class B) level.  EDIT: There are now also "advanced" (1800-2000) and "master path" (2000-2200) level study plans listed.
  • Chess Training for Budding Champions by Jesper Hall, despite the title, is a rather advanced book on training designed for the ambitious amateur.  It does well in providing concrete ideas for setting up training programs, illustrates topics well with annotated games, and was the primary inspiration for my own focus on analyzing your own games.
  • More recently, Axel Smith's Pump Up Your Rating is another good single-volume work, with perhaps more focus on teaching the concepts rather than illustrating them, yet with training in mind.  I believe it's worth quoting an excerpt from the introduction, which echoes some of the themes I've addressed in this blog:
"[Serious chess training] can be summarized in five words: active learning through structured training...The four pillars of chess training described in this book, in decreasing order of importance, are:
  •  The List of Mistakes - analyzing your own games and categorizing the mistakes.
  • The Woodpecker Method - learning the tactical motifs and solving simple exercises to internalize them into your intuition.
  • Openings - studying them in such a way that you also learn middlegame positions and standard moves.
  • Theoretical Endgames - studying them only once."
My own eclectic program has a better chance of being followed this year, as I expect to have more time to concentrate on my chess.  What I would like to do is come up with a rotating list of chess resources (books, DVDs, significant videos/articles, etc.) that I complete fully before re-populating the list.  The list should be broad and contain opening, middle and endgame works, along with other general improvement topics.  Regular tournament-level play (online and OTB) and game analysis (both my own and master-level commentaries) will continue in parallel, along with tactics training via Chess.com's Tactics Trainer and the Chess Tactics Server.  I'll put up the next list once I have it together, as reference and to keep me honest by having a public goal.

No quick fix

Any improvement program requires significant time, dedicated effort and serious study.  The chess world is blessed with a large volume of books, software programs and online resources that allow us to make progress; in the end, however, it's up to us to do the necessary work (and play).  I would like to finish with the following quote from Grandmaster Versus Amateur, however, which gives a useful perspective:
"One happy aspect of being an amateur is that one can remain focused on the artistic side of the game, by not being in a permanent rush to win prizes and rating points."
So let's keep in mind the positives about being amateurs!

10 January 2015

Chess computing resources for 2015 - Part III: Game databases and e-books


In this last part of the series - following Part I on database analysis programs and Part II on chessplaying software - I'd like to look at some of the database resources available and discuss their importance for the improving player.  In addition to standard database files, typically containing very large collections of games in a proprietary or PGN format, there are now online, browser-accessible database references and a large number of e-books which are in reality database files with associated text and annotations.  We'll look at some examples from each category.

Database files

The first indispensable database for any player is one that contains your own games.  Unless you've been keeping separate online records of your games - something many chessplaying sites will now do automatically for you - you will need to manually enter them on your database program board.  This in itself can be a learning experience, so even if you have a stack of old tournament scoresheets lying around, that can be viewed as an opportunity rather than a chore.

I keep separate databases for my over-the-board (OTB) tournament games and slow online games completed at Chess.com (mostly from the Slow Chess League), since I prefer to split the different types of games.  As needed, though, I could easily combine them into a single database by copying and pasting the games into a new one.  The importance of having a personal games database is hard to underestimate, although I think many amateurs often do just that.
  • For the improving chessplayer, at minimum all of your tournament games should be recorded and analyzed to determine what you did right, as well as where you went wrong, and what general and specific lessons you can take away from each game.  Aside from games decided by rare one-move blunders in the opening - which may also be worth remembering for the future - each game then becomes an opportunity for in-depth chess study and research.  See "Analyzing your own games is more than analyzing your own games" for more on this approach to training.
  • Personal databases allow you to generate statistics on your performance at the push of a button.  Even basic stats such as your won-loss record with each color, score and performance rating by opening, and score against different rating categories can lead to new insights and identify unexpected weaknesses (or strengths).
  • Your personal database should be one of the centers of your preparation for future tournaments/games, especially if you have regularly recurring opponents, opening lines, or middlegame and endgame themes.  Understanding and identifying commonalities in past games, including solutions that you reached in subsequent analysis, should guide your expectations for future play.  This helps ensure that the lessons you learn one day are not simply forgotten the next.
Next we have what most people automatically think about when they see the word "database", the very large comprehensive databases with millions of games.  These types of reference databases are fundamental for analysis, research and comparison purposes.  Along with quantity of games, however, quality of games is just as important, and having annotated games included is a major benefit.
  • For ChessBase users, the yearly "Mega" database is the gold standard, primarily because it includes a significant number of games annotated by world-class players.  You can easily search on just annotated games, for example, to get the most out of this feature.  The ChessBase "Big" database version is similarly large, but lacks the annotated games.
  • The other main commercial option, Chess Assistant, comes with a database with a similarly high number of games (over 6 million).
  • Free large-scale databases for download can be found via internet search.  I can't vouch for their quality, but one good place to start for resources is Chessopolis.  To a large extent the freeware function for reference databases has been taken over by browser-accessible databases (see next section).
  • Updating your main reference database can be done either commercially - both of the commercial packages mentioned above typically come with one year of free downloaded updates - or via The Week in Chess (TWIC), an outstanding freely available service to the international chess community.  TWIC has other news and features that are of interest.
The last category of databases I want to discuss is specific to a purpose and typically is created by a player interested in a particular topic, using a search on a reference database.  Some other prepackaged databases also fall into this category, including ones specializing in a particular opening, for example.
  • One of the most obvious "purpose" databases to create is one containing the games of a known future opponent.  Even for amateurs this can be easily done in online communities, where individuals' games are accessible and often downloadable in PGN.  I do this at Chess.com when preparing for Slow Chess League games, to see what openings my opponents use and to see something of their style of play.  Professionals of course do this on a much more intensive level.
  • One opening study method is to look at the games of a model player, someone who frequently played the opening with success on a professional level.  A database search, typically by ECO code, can easily create such a roster of games for review and analysis, as I did with the "Stonewall Hero" exercise.  IM Jeremy Silman also discusses the use of databases for opening preparation in a recent Chess.com article.
  • Specific databases for study purposes can be created based on almost anything that you can think of, including specific middlegame and endgame structures, subject to the limitations of the search functions in your database analysis software.
  • This blog includes PGN database collections of my published annotated commentary games (master-level) and personal games; download links are in the sidebar.
  • Blunder Prone has some recent comments on deliberate practice using databases.

Online databases

Browser-accessible, large online databases have proliferated and evolved to the point where they have become a genuine alternative to prepackaged, local database files.  Often they will allow you to download your search results in PGN format, which allows you to then annotate and manipulate the databases for study.  Because of the large number of database resources now available, I won't try to provide a comprehensive list, but rather a few selected places to start.
  • Chess Tempo is a site whose primary function is to serve as a training resource, with both basic (free) and commercial features.  For example, it has tactics training, endgame training, and a range of options for computer opponents (the topic of Part II).  However, it also has an online, updated databases of 2+ million games with advanced search and filter options.  Laurent S mentioned this in his comment on a previous post on computer resources.  
  • 365Chess.com is another site featuring a large (3.5+ million) searchable database that also has some training functions.  It is set up similarly with basic (free) and advanced (commercial) features, with PGN downloads and database creation among the latter.
  • On a simpler level, the Shredder site has opening and endgame online databases that also allow you to paste in a position (in FEN) for searching.
  • While Chess.com is not primarily a database site, its "Game Explorer" feature allows for basic database exploration and PGN download of games.
It's worth noting that the main commercial database products also now have "live" components, with ChessBase's LiveBase accessible to its customers.


E-books

The more sophisticated e-book products are, in essence, "purpose" databases where the main selling points are the narratives/text and annotations included with the games, rather than the games database per se.  This is not to say that simpler e-books, which are essentially digital copies of print editions, aren't useful; they significantly enhance both the portability and availability of chess products.  However, e-books that contain databases have a significantly greater potential for your own investigation and analysis, as you can more easily explore related ideas via included links and your own searches.  They also don't require you to have a physical chess set with you.
  • Everyman Chess is a leading e-book publisher and their extensive free sample downloads are an excellent way to "try before you buy".
  • ChessBase has a similarly wide variety of "Fritztrainer" training products that include video narration along with annotated databases.  The DVDs can be used either with a commercial ChessBase product or with a free viewer.  One of the first I purchased was Viktor Kortchnoi's autobiographical "My Life for Chess" which was both entertaining and useful.
  • While opening study is an obviously useful subject for e-books, they are also taking advantage of the format's applicability to everything from game collections of top players to classic training works like Capablanca's Chess Fundamentals.

What about training software?

While I've included references to a number of training resources and methods in this post and in the earlier parts of this series, I haven't looked at computer resources whose specific function is training.  I will, however, include some discussion of them in an upcoming post - "Chess Improvement Programs: Directed vs. Eclectic".

03 January 2015

Chess computing resources for 2015 - Part II: Chessplaying software and computer opponents


In contrast with Part I of this series, which looked at more specialized database analysis programs, Part II covers a greater variety of computer chess tools that include chessplaying software and computer opponents.  Many software packages include all functions - database, playing, analysis, and training - but typically focus on one category more than others.  Because of the great number of differences in features between different software packages, in this post I'll focus on common chessplaying functions and how an improving player can take advantage of them.  After that discussion is a review of some computer resources, including freeware and online.  A brief overview of popular engines is also included.

As a side note, it's been interesting for me to see how chess software has evolved away from dedicated chessplaying programs - an approach exemplified by the old Chessmaster series - to a more modular approach, one which splits the GUI (graphical user interface) and the chess engine.  Users now can easily mix-and-match, using their favorite engine(s) along with whatever engine is bundled with the software they use.  This splitting of functions has its pluses and minuses, however, as I think some of the focus on the actual playing experience has been sacrificed.  The performance differences between different engines also are not nearly as significant as they used to be, especially for chessplaying purposes. Personally, I've found it difficult to take computer opponents seriously, as I have trouble caring about the outcome of a game when it is not at least somewhat similar to tournament conditions.

Computer opponents - what are they good for?

All chessplaying software packages offer different options for handicaps, in which the engine's effective playing strength is reduced.  Different mechanisms are used to accomplish this, with varying results.  The simplest method is to use a fast time limit for the computer to move (1 second, 5 seconds, etc.)  Another simple weakening method is to not permit the computer to "think" on its opponent's time.  For the more basic engines and those used for online play in browsers, these are popular choices (sometimes the only choices) to help calibrate the computer opponent's strength.  For an improving player, I think using the quick time method can be detrimental for training, since it psychologically encourages you to think and move fast as well, rather than develop and employ a comprehensive and coherent thinking process.  However, if what you want is a casual game within a quick time limit, or are using blitz games for a specific training purposes such as opening familiarization, then fast time controls are certainly appropriate.

This brings us to the question of what computer opponents are good for, within the context of a training/improvement program.  I think that they are best viewed as "sparring partners" - a concept familiar from other individual combat sports, boxing being the most well known, but also including fencing and martial arts.  Good sparring partners should offer you several things:
  • The chance to get a workout - a mental one in the case of chess - allowing the player to practice and reinforce their skills and thinking habits under conditions similar to that of a real match.  This usually requires a "slow chess" time control (45 45, 60 5, or higher).  Since computer sparring partners will not care what the time control is set at, having nothing else to do, this is a bonus - as long as you have the necessary focus and patience yourself.
  • A level of opposition that is at or (preferably) somewhat above your own playing strength.  You should have a legitimate chance to beat your sparring partner, albeit with significant effort.  Running an engine at top strength would prove a frustrating and insurmountable hurdle, while beating up an overly handicapped opponent will teach you nothing.  This concept is related to effortful study, a process in which you constantly push your boundaries, but without taking on tasks completely beyond your strength.  In the context of a computer opponent, once you start beating them more than 50 percent of the time, you should increase their strength.
  • Opposition that has different "styles" or combinations of strengths and weaknesses.  It takes different skills to face and defeat an opponent who always goes for aggressive attacks, versus one who is content to play quietly and set up a solid position.  Opening variety is also important, to become familiar with different position-types and avoid being surprised.
How you set up your "sparring partner" therefore will greatly affect your experience.  Your chessplaying software should have a variety of slower time controls (often which you can input yourself), handicap methods that you can understand and easily adjust, and ways of varying the computer opponent's play.  These latter two points deserve some additional attention, especially in the context of their training usefulness.  
  • Despite the wide variety of handicap settings available, computer opponents often will play in an unrealistic manner that detracts from the experience and makes training less effective.  For example, they may randomly decide to drop material, make obviously pointless moves, and the like.  This is unfortunately hard to avoid from the programmer's point of view when trying to find practical ways to make a program play weaker than it can.
  • Computer opponents without randomizing functions will play the exact same game every time, which is of very limited use for training (and not very enjoyable, either).  Make the computer opponent behave too randomly, however, and it becomes difficult to learn via iterative training and amass useful experience, especially with mainline openings and key common position-types.
Each person will have their own particular preference for what they are looking for in a computer opponent.  That said, I'll offer some practical guidelines:
  • If the software package has automated functions that address the above issues, use them.  For example, some offer a specific Elo rating setting for opponent strength.  In that case, you can set the opponent to be 100-200 points higher than your own rating and see how you do against it over a few games, then adjust the strength in future games as necessary.
  • The Fritz GUI in addition offers a "Sparring" mode for new games, with five level settings (from "Very Easy" to "Really Hard").  The program calibrates its play accordingly and will deliberately play a certain number of moves, based on the level setting, for which it can calculate a tactical refutation, but otherwise play a strong game.  I find this mode of play to be more realistic in simulating a "real" game, as the program does not simply give material away (or allow itself to be mated) without requiring significant effort from its opponent.
  • When a default opening book is included with a software package, there are usually settings that will force the computer to play "best" or "optimal" lines, or alternatively force more variety in the opening choices.  It takes additional work, but can be even better for training purposes, when you edit or assemble your own opening book for the program to use.  Naturally using your own repertoire database is a possibility for this.
  • If you are doing tactical or visualization drills, then using extra features in a GUI such as displaying hints, threatened squares, etc. may be useful.  Naturally none of these aids are available during tournament conditions, so I would suggest not using any of these for serious training games.
Computer resources and packages

The following list is not meant to be comprehensive, but rather to give a sampling of the major programs available and types of resources, including online freeware, that can be found.  Comments on these, and on resources not originally included, would be useful to see; I'll incorporate them as appropriate in future edits.

The Deep Fritz 14 GUI is the latest version of the long-running Fritz series from ChessBase.  It has a full range of features for serious play, including a large number of different 2D and 3D virtual chess sets, as well as advanced analysis functions and some database functions.  For players looking for a high-end sparring partner and superior analysis capabilities, along with basic database resources, this GUI would be a good starting package.  For more advanced database functions and software, see Part I.  It's worth noting that ChessBase sells the GUI under the name of the engine bundled with it, so for example the latest Komodo 8 product is simply the Deep Fritz 14 GUI with Komodo 8 included as the primary engine.  Other UCI engines (see below) can be added easily to the GUI.

The Aquarium program - the latest release being Aquarium 2015 (and an additional version with Houdini 4 bundled) - is another chess package that includes a full range of analysis and playing functions, although its GUI and functionality is built primarily around the analysis functions.  Published by ChessOK, its database also uses the same proprietary format as their Chess Assistant database software.  Aquarium 2015 currently appears to have some issues with its deep analysis (IDeA) functions.

Shredder - as mentioned by Tommyg in his comment on Part I - remains a commercial alternative GUI, notably being available across mobile platforms as well, and has some online browser-accessible features on the site.  The playing community seems to be a strength of the product. Hiarcs Chess Explorer is another alternative all-in-one software package (thanks also to Tommyg for the comment on it).

The Chessmaster series, as mentioned above, was focused primarily on the playing experience, wth Chessmaster XI (also known as Chessmaster: Grandmaster Edition) being the last edition.  I previously recommended it as a sparring partner, since it had by far the best range of customizable opponents, as well as an outstanding video tutorial series included with it.  Unfortunately this product (from 2007) is no longer supported by software publisher Ubisoft, so I can't recommend it anymore.  I was originally able to install it on my Windows 7 machine with some difficulty, but eventually after one Windows Update cycle the program refused to work properly, even after reinstalling it multiple times.  If you have an older machine, it may still function.

The SCID vs PC freeware database analysis package was mentioned in Part I and it also includes some basic chessplaying functions.  There are other free GUIs available that are more developed around the playing and analysis functions, for example the more complex Arena Chess GUI 3.5 or the simpler Tarrasch Chess GUI.

Chess.com has a browser opponent available, or you can play against different bot opponents in its live section.

Engines

I've saved the section on engines for last, in part because for playing purposes they are now so strong that differences among top engines are essentially irrelevant.  Your choice of handicap options will be much more important in determining the type of playing experience you have, rather than which Elo 3000+ engine your computer is running.

For analysis purposes, engines may still show some significant if not large differences between them, at least when there are no concrete tactical advantages.  So it is from the analysis use perspective that I'll offer a few comments, along with the top engine home page links.  All of them can be downloaded directly from the links in their UCI versions, which you can then plug into your preferred GUI.  As an alternative, some intentionally weak engines have been developed for playing purposes - they may be included in some software packages - and may prove useful as sparring partners without having to handicap them.
  • Komodo 8 is what I currently use as an analysis engine.  It has an interesting history and appears to incorporate a greater degree of human positional judgment than its peers, as discussed in this ChessBase news article.  It is this latter quality that was the deciding factor for me and it seems legitimate.  In using Komodo 8, I've noticed that it gives (slightly) greater evaluation weight to compensation for sacrificed material and other positional factors than other engines; computers historically have been too materialistic, although that's changed to a great extent.
  • Houdini 4 is the latest version of the Houdini engine series.  I used the (freeware) Houdini 1.5a and (commercial) Houdini 2.0c previously for analysis engines.  Houdini appears to be somewhat faster than Komodo 8 in its evaluation functions, which remain top-notch.
  • Stockfish (currently version 5) is the top freeware UCI engine and can hold its own at the Elo 3000+ level.  Using this as a freeware option means you are sacrificing little (if any) power and functionality and it has downloads for a variety of platforms, including mobile apps.
In Part I, I discussed the practical value of using engines for game analysis, as part of an integrated process involving database analysis software (and of course your own brain).  This is a legitimate use of a tool to make progress.  What should be avoided is the practice of using software to replace (rather than augment) your brain, as sometimes occurs when people simply pop their games into a "full analysis" software feature, let it run, briefly see what the analysis results are, then move on.  A more detailed look at the pros and cons of computer analysis can be found in "Pitfalls of Computer Analysis" on this blog.  A more exaggerated (if entertaining) view of the harm relying on computer engines can be found in IM Jeremy Silman's recent article on Chess.com.  Coincidentally, a recent article on the "Rise of the Machines" at The Chess Improver also touches on these and some other related issues.

The future

All of these programs will no doubt continue to evolve over time, with new entries in each subcategory.  I think from a functional perspective the logical path would be to combine top-level database, analysis and gameplaying functions in a single software suite.  ChessBase could do this now with relative ease, I suspect, but keeping ChessBase and Fritz products separate is a way of segmenting the market for additional revenue.  ChessOK has a similar type of split with its Chess Assistant and Aquarium products.

All of the major packages (even freeware) now include database analysis functions, so deliberately keeping major playing/analyzing functionality separate from database functions appears more and more artificial.  I'm still hoping, even after a poor experience with the first version of the Chess King program, that a single and economical software package will eventually emerge that has an aesthetic GUI, engaging and varied computer opponents, good analysis functions, and powerful database options.  With the engines now separated from the GUIs and at a very high strength level, perhaps programmers can turn more attention to the gameplaying experience and thinking through how best to present and display analysis options and results.

Given the quality of the freeware available today, I recommend that people try out different available programs before purchasing a new commercial package.  Even if you do decide to go the commercial route, you'll have (for free) a better idea of what features you desire most, and/or what you really don't like, something which I think is central to effectively using computer tools in a training program.

The next post in this series is Part III, which looks at database and e-book resources.