tag:blogger.com,1999:blog-36142036889512584632024-03-13T05:15:14.945-07:00Time Series SoftwareNotes and short articles about <a href="http://agent.ch/timeseries/crnickl/">CrNiCKL</a> (chronicle) and the <a href="http://agent.ch/timeseries/t2/index.html">Time2 Library</a>.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.comBlogger24125tag:blogger.com,1999:blog-3614203688951258463.post-2448423610765197452017-10-26T11:25:00.001-07:002017-10-26T11:25:55.797-07:00Version 2 of the Time2 Library and CrNiCKLNew versions of the Time2 Library and of CrNiCKL have been released on October 5, 2017.
<p>
The software can be found on <a href="https://github.com/jpvetterli/">GitHub</a> and in
<a href="http://repo.maven.apache.org/maven2/ch/agent/">Maven Central</a>.
Various resources are available on the
<a href="http://agent.ch/timeseries/t2/">Time2 Library project website</a> and the
<a href="http://agent.ch/timeseries/crnickl/">CrNiCKL project website</a>.
<p>
Most changes are in the Time2 Library and can be summarized as:
<ul>
<li>improved thread safety, achieved by the elimination of
<code>TimeDomainFactory</code>,
<code>TimeSeriesFactory</code>, and
<code>ExternalTimeFormat</code>;
<li>partial reorganization of the package structure;
<li>support for nanoseconds;
<li>various fixes and improvements.
</ul>
Consult the <a href="http://agent.ch/timeseries/t2/RELEASE_NOTES.html">release notes</a>
for more details.
<p>
Version 2.0.0 of the Time2 Library is not fully compatible with version 1.1.7. Due to the elimination
and replacement of some classes and interfaces, a minimum of application recoding is unavoidable.
It is in principle limited to initialization code. In addition, the reorganization of packages makes
it necessary to reorganize imports.
<p>
The important change is the improvement of thread safety. Factories were implemented as
singletons and thread safety was enforced with the help of <em>synchronized</em> methods. In
rare and hard-to-replicate cases, the software experienced deadlocks. To address the issue, all
singletons have been eliminated. The functionality is of course still available. It is implemented by
using injection and immutable objects. The new approach does not require any synchronization and
is inherently safer. As a bonus, it also makes testing easier.
<p>
Instead of <code>TimeDomainFactory</code>, applications use a <code>TimeDomainCatalog</code>,
implemented as an immutable object with all time domains passed as parameters to the constructor.
The role of <code>TimeSeriesFactory</code> has been taken over by public constructors of
<code>RegularTimeSeries</code> and <code>SparseTimeSeries</code>.
The functionality of <code>ExternalTimeFormat</code> has been split into <code>TimeScanner</code>
and <code>TimeFormatter</code>.
<p>
The CrNiCKL database software depends on the Time2 Library and has been upgraded. Its version 2.0.0
is likewise incompatible with the previous one, 1.2.1. The JDBC and MongoDB drivers have also
been upgraded. Use crnickl-jdbc 2.0.1 and crnickl-mongodb 2.0.1 instead of 2.0.0; the 2.0.0 versions
have a glitch in their POMs and cannot be used for unit testing.
<p>
The demos on GitHub have been updated and provide a simple illustration of how
to use the new versions. The direct links are
<a href="https://github.com/jpvetterli/time2lib-demo">time2lib-demo</a>
and <a href="https://github.com/jpvetterli/crnickl-demo">crnickl-demo</a>.
Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-9153070861518827442013-05-01T08:57:00.000-07:002013-05-01T08:57:25.473-07:00New CrNiCKL version addresses issue with JUnit and Java 7Version 1.2.1 of <code>crnickl</code> was released yesterday April 30, 2013.
It comes with a small improvement to the API and addresses a problem with unit tests under Java 7
<p>
<b>API improvement</b>
<p>
The <code>Database</code> interface declares a new method named <code>makeSurrogate</code> to
turn a string into a <code>Surrogate</code>. The string is typically produced by a surrogate's <code>toString</code> method. It was previously not possible to create a surrogate from such a string without making some assumptions and using low-level methods.
<p>
<b>Issue with JUnit and Java 7</b>
<p>
JUnit tests written with JUnit are usually short pieces of code which can be run independently of each other.
However, when writing test cases for a database system, it is often handy to assume that test methods are executed in a predictable order, for example in source code order. Under this assumption, a first series of test methods populates the database, followed by a series of methods which modifies the data, and finally a series of methods which deletes the data.
<p>
Making this assumption was fine until upgrading to Java 7. The new implementation of the reflection API differs from its predecessors in such a way that JUnit now executes test methods in seemingly random order. This wreaks havoc when the order of tests is important.
<p>
To address this issue the <code>@FixMethodOrder</code> annotation has been introduced in JUnit 4.11. It can be used to force execution of test methods in alphabetical order.
<p>
Version 1.1.0 of <code>crnickl-test</code> uses this new annotation and renames some methods so that source order and alphabetical order agree. <code>crnickl-test</code> is a library providing common tests for all database drivers and is used by <code>crnickl-jdbc</code> and <code>crnickl-mongodb</code>.
<p>
The software can be found on <a href="https://github.com/jpvetterli/">GitHub</a> and in
<a href="http://repo.maven.apache.org/maven2/ch/agent/">Maven Central</a>.
<p>
<a href="http://agent.ch/timeseries/crnickl/">CrNiCKL project website</a><br/>
Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-68072026661900675262013-01-22T04:22:00.000-08:002013-01-22T04:22:58.954-08:00CrNiCKL driver for MongoDB released A CrNiCKL driver for <a href="http://www.mongodb.org/">MongoDB</a>
was released yesterday. With this first NoSQL driver, developers have now
the choice to run CrNiCKL services either on SQL or on NoSQL databases. A NoSQL
database can be a good solution for accommodating huge datasets or to address
replication or <a href="http://en.wikipedia.org/wiki/Sharding">sharding</a> requirements.
On the other hand, when <a href="http://en.wikipedia.org/wiki/ACID">data integrity</a> is paramount, it
is better to stick with a good old SQL database and use the CrNiCKL driver for JDBC.
<p>
The drivers can be found on <a href="https://github.com/jpvetterli/">GitHub</a> and in
<a href="http://repo.maven.apache.org/maven2/ch/agent/">Maven Central</a>.
<p>
<a href="http://agent.ch/timeseries/crnickl/">CrNiCKL project website</a><br/>Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-47049515371325302562012-09-08T14:45:00.000-07:002012-09-08T14:46:41.251-07:00The Time2 Library and CrNiCKL now in Maven Central The projects artifacts are now deployed to Maven Central.
It is not necessary to specify a repository in the settings or
POM files any more. Using this software with maven is now
100% straightforward.
<p>
<a href="http://agent.ch/timeseries/t2/">Time2 Library</a><br/>
<a href="http://agent.ch/timeseries/crnickl/">CrNiCKL database</a><br/>
</p>Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-86061958979455503002012-09-06T06:46:00.000-07:002012-09-08T14:50:09.260-07:00Is it broke? Can we fix it?<small>This is an update to a
<a href="http://timeseriessoftware.blogspot.com/2011/10/why-make-it-complicated-when-you-can.html">post</a> from last year. The post itself has not been updated.
You can find the complete article on <a href="http://agent.ch/articles/DateAndTimeMess.html">agent.ch</a>.
</small>
<br/>
<br/>
Experts met on January 19, 2012 at the International Telecommunication Union
to decide whether to abolish leap seconds. Due to a lack of consensus among
participants it was decided to postpone the decision
(<a href="http://www.bbc.co.uk/news/science-environment-16625614">BBC News</a>).
It was a classical standoff between those who want sharp, systematic
solutions and the advocates of
<q><a href="http://en.wikipedia.org/wiki/Wikipedia:If_it_ain%27t_broke,_don%27t_fix_it">if
it ain't broke, don't fix it.</a></q> A few months later, on June 30, 2012,
yet another leap second was added, causing problems at some websites.
<em>Wired</em> reported it under the dramatic sounding heading
<q><a href="http://www.wired.com/wiredenterprise/2012/07/leap-second-glitch-explained/">The
Inside Story of the Extra Second That Crashed the Web</a></q>.
The discussion is open: is it broke? Can we fix it?
Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-56804474576851167312012-08-29T01:45:00.000-07:002012-09-08T14:44:07.119-07:00The Time2 Library and CrNiCKL support MavenYesterday, I moved some of my Java projects to <a href="http://maven.apache.org">Maven</a>.
There is no difference from a client point of view. The new JARs are plug-compatible
but not identical, so checksums have changed. This is the only reason why minor version numbers have been +=1d.
<p>
The software is not yet available from <a href="http://repo.maven.apache.org/">The Central Repository</a>,
but can be found in a freely accessible maven repository on <a href="https://github.com/jpvetterli/maven-repo/">GitHub</a>.
To use it, specify this in your POM or settings:
<small><pre><code><repository>
<id>jpvetterli on github.com</id>
<url>https://raw.github.com/jpvetterli/maven-repo/master/releases/</url>
</repository>
</code></pre></small>
<small>
Links:
<br/>
<a href="http://agent.ch/timeseries/t2/">Time2 Library project</a>
<br/>
<a href="http://agent.ch/timeseries/crnickl/">CrNiCKL database project</a>
</small>
Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-34720871537160508342012-07-17T14:45:00.000-07:002012-07-17T14:45:26.422-07:00CrNiCKL 1.1.0 released with all delete methods renamedThere were quite a few methods named <code>delete</code> in the CrNiCKL software.
This was a bad idea because such methods are useless when called from JavaScript.
Indeed, <code>delete</code> is a reserved word in js.
<p>
All such methods have been renamed and
a new version of the software has been released to the
<a href="http://agent.ch/timeseries/crnickl/">project website</a>,
to <a href="https://sourceforge.net/projects/crnickl/">SourceForge</a> and the <a href="https://github.com/jpvetterli">GitHub repositories</a>. The following scheme
has been used for renaming. Parameterless <code>delete</code> methods are now
named <code>destroy</code>. The others are now typically named <code>deleteFoo</code>
where <code>Foo</code> is the type of (one of) the parameter(s).
<p>
I decided to completely eliminate the old methods rather than let them live a bit longer as deprecated. I did it because the software is not yet used out there. In case I misread the stats, please accept my apologies. In any case the previous version remains available on all the websites mentioned.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-63630134410088680892012-07-13T10:38:00.000-07:002012-07-13T10:38:28.214-07:00Source code sharingSince a very long time I wanted to switch to the <a href="http://git-scm.com/">git</a> version control system. It's now done. Compared to what I've used in the past: cvs, sccs, rcs, vss, sclm, Librarian, and maybe others I have forgotten, I like git better. I don't know why I waited so many years...
<p>
Although the source of the projects I discuss in this blog was already available for downloading via the project websites,
it can now be browsed more comfortably at <a href="https://github.com/jpvetterli">GitHub</a>.
<p>
The relevant repositories are:
<p>
<dl>
<dt><a href="https://github.com/jpvetterli/time2lib">time2lib</a></dt><dd>for the Time2 Library</dd>
<dt><a href="https://github.com/jpvetterli/crnickl">crnickl</a></dt><dd>for the CrNiCKL base system</dd>
<dt><a href="https://github.com/jpvetterli/crnickl-jdbc">crnickl-jdbc</a></dt><dd>for the JDBC support of CrNiCKL</dd>
<dt><a href="https://github.com/jpvetterli/crnickl-demo">crnickl-demo</a></dt><dd>for the CrNiCKL demos</dd>
</dl>Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-39195293491460353802012-07-13T10:02:00.000-07:002012-07-13T10:03:17.990-07:00The game of the nameThe ChronoDB project has been renamed CrNiCKL. I noticed a naming conflict with an unrelated product and decided to change the name when it's still harmless.
<p>
Chronicles are the most important objects in the <del>ChronoDB</del> CrNiCKL database. So I started from them to find a new name. I settled on CrNiCKL because it can be pronounced like "chronicle" and there are practically no hits for it in Google. I also decided to write the name with most of the letters in uppercase, so it stands out in text.
<p>
The software remains available under its old name, for a while at least, but won't be updated. Blog posts related to ChronoDB will not be updated but will get a notice about the rename.
<p>
The new project website is <a href="http://agent.ch/timeseries/crnickl/">http://agent.ch/timeseries/crnickl/</a>.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-51492840299907185982012-07-11T01:55:00.000-07:002012-07-13T10:03:33.839-07:00Time2 Library project home page updatedThe marketing departement has just updated the
<a href="http://agent.ch/timeseries/t2/">home page</a> of the Time2 Library project with
the following <q>quote of the day</q>:
<p>
<b><blockquote><q>
If you need speed, compactness, and flexibility, then the Time2 Library is
for you. There is no trade off. Even with the wildest time domains
(calendars if you prefer) a time point is just a number.
In many cases it is even less than that: it is an array index.
The flexibility comes into play only if and when needed,
for example when printing or scanning a date.
</q>
</blockquote></b>
Wow.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com2tag:blogger.com,1999:blog-3614203688951258463.post-17221615060028442662012-07-09T15:08:00.000-07:002012-07-13T10:15:57.260-07:00ChronoDB and geographical coordinates<hr>
<b>
Important notice. On July 13, 2012, the ChronoDB project was renamed CrNiCKL, which is pronounced like "chronicle". All packages, demos included, have been renamed. The new project website is at <a href="http://agent.ch/timeseries/crnickl/">http://agent.ch/timeseries/crnickl/</a>. The old project remains accessible for a while
at <a href="http://agent.ch/timeseries/chronodb/">http://agent.ch/timeseries/chronodb/</a>. The remainder of this article remains valid <em>mutatis mutandis</em>.
</b>
<hr>
<p>
I've just released another ChronoDB demo to show how to set up a
database with <b>time series of geographical coordinates</b>.
The demo can be found in the package <small><code>ch.agent.chronodb.demo.geocoord</code></small>
in archive <small><code>chronodb-demo-1.1.0.jar</code></small>
at the <a href="http://agent.ch/timeseries/chronodb">ChronoDB project</a> website or on <a href="http://sourceforge.net/projects/chronodb/files/chronodb-demo-1.1.0.jar/download">SourceForge</a>.
<p>
GeoCoord is a toy Java interface for geographical coordinates, with three methods:
<small><xmp style="background: white">boolean isNear(GeoCoord coord);
boolean isNear(GeoCoord coord, double distance);
double distanceTo(GeoCoord coord);
</xmp></small>
In the demo, GeoCoords are implemented as <a href="http://en.wikipedia.org/wiki/Coordinates_(geographic)#Cartesian_coordinates">
cartesian coordinates</a>. The glue with ChronoDB is provided by a simple value scanner <small><code>GeoCodeValueScanner</code></small> and a less simple implementation of
<small><code>ValueAccessMethods<GeoCoord></code></small> named
<small><code>AccessMethodsForGeoCoord</code></small>. The <small><code>Database</code></small> class itself is so
small it can be listed completely here:
<small><xmp style="background: white">package ch.agent.chronodb.demo.geocoord;
import ch.agent.chronodb.jdbc.JDBCDatabase;
public class GeoCoordDatabase extends JDBCDatabase {
public GeoCoordDatabase(String name) {
super(name);
setAccessMethods(GeoCoordValueScanner.class.getName(),
new AccessMethodsForGeoCoord());
}
}
</xmp></small>
<p>
To make things interesting the demo uses a special time domain, with
time points at 7:00 AM, 9:00 AM, 3:11 PM, and 9:33:20 PM every Monday, Tuesday,
Friday, and Saturday. For a refresher on time domains, please
have a look at my <a href="http://timeseriessoftware.blogspot.ch/2012/07/time-series-framework-design.html">previous post</a>.
<p>
Here is what running the demo looks like (commands are on a single line and
output has been truncated):
<small><xmp style="background: white">$ jar xf chronodb-demo-1.1.0.jar lib
$ java -cp chronodb-demo-1.1.0.jar \
ch.agent.chronodb.demo.geocoord.GeoCoordDemo Resources/geocoord.parm
2012-04-09 09:00:00 8151km (machin)
2012-04-09 15:11:00 7000km (machin)
--- clip clip ---
2012-07-09 07:00:00 7053km (truc)
2012-07-09 09:00:00 8367km (bidule)
2012-07-09 15:11:00 9582km (bidule)
2012-07-09 21:33:20 6889km (machin)
$
</xmp></small>
The exact figures vary with each run because the data is random.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-76419955792800258122012-07-06T16:19:00.000-07:002012-07-09T01:16:44.286-07:00Time Series Framework DesignThis note is an edited summary of the ideas underlying the design of the
<a href="http://agent.ch/timeseries/t2/">Time2 Library</a>.
<p>
With time playing a rather important role in this world,
sequences of things ordered by time are present in many kinds of systems.
Because the idea of time is intuitively familiar, it is tempting to choose
a simplistic design when modeling a system or even not to think about it at all.
Ironically, simplistic designs can lead to needlessly complex implementations,
as annoying issues are addressed one after the other.
<p>
Stated informally, a time series is a set of elements uniquely identified
by a discrete point in time or by a time interval.
The Time Series Framework requires a more precise definition:
<blockquote>
<em>A time series is characterized by a value type and a time domain.
All elements of a time series have a value of the same type, the value type,
or can be recognized as missing. Any element can be uniquely identified
by a point in the time domain of the series.
</em>
</blockquote>
Note that the definition does not explicitly allow for time ranges identifying
values. This is generally not a problem because of the nature of time domains.
<p>
The terms used in the definition will be explained shortly, but before
that a few typical problems will be discussed. These problems should
illustrate why a framework for time series is useful.
<p>
<hr>
<h3>Problems and design goals</h3>
Let us look at a series alternating between two
constant values every working day: Friday=5, Monday=10, Tuesday=5,
Wednesday=10, Thursday=5, Friday=10, Monday=5, Tuesday=10, and so
on. The series has never any data on weekends. Consider
this straightforward chart plotting the values on the
vertical axis against their dates, on the horizontal axis:
<p>
<table>
<tr>
<td><a href="http://agent.ch/images/chart_week_ends.svg"><img style="border:0"
src="http://agent.ch/images/chart_week_ends.png"/></a></td>
</tr>
<!--tr>
<td align="center"><small>Chart 1</small></td>
</tr-->
</table>
This looks a bit funny, doesn't it? The chart does not express very well the
perfect regularity of the series. Compare it to
this <em>designed</em> solution:
<table>
<tr>
<td><a href="http://agent.ch/images/chart_no_week_ends.svg"><img style="border:0"
src="http://agent.ch/images/chart_no_week_ends.png"/></a>
</td>
</tr>
<!--tr>
<td align="center"><small>Chart 2</small></td>
</tr-->
</table>
The visual pattern is now a faithful expression of the regularity of
the data. The chart does it by excluding weekends. The <strong>time
domain</strong> is in fact the only difference between the two charts:
everyday dates in one case versus workweek-only dates in the other.
<p>
<strong>Tel Aviv, Cairo, New York</strong>
<p>
Beyond this contrived example, there are many situations with
no data on weekends. A familiar case is provided by stock markets.
These are also a good illustration of the annoying details hiding in
seemingly simple problems: weekends are not the same in Tel Aviv,
Cairo, and New York. When designing a database for global stock market
data, or when drawing charts to compare the prices of some American,
Egyptian, or Israeli stocks, such issues must be dealt with.
<p>
Getting a good grip on the time domain is the <strong>first important design
goal</strong> of the Time Series Framework.
<p>
<strong>Missing values</strong>
<p>
The following table lists the first <em>eleven</em> Olympic Games.
<p>
<table style="background: white; margin-left:20px" cellpadding="3">
<tr><td>1896</td><td>Athens</td></tr>
<tr><td>1900</td><td>Paris</td></tr>
<tr><td>1904</td><td>Saint-Louis</td></tr>
<tr><td>1908</td><td>London</td></tr>
<tr><td>1912</td><td>Stockholm</td></tr>
<tr><td>1916</td><td></td></tr>
<tr><td>1920</td><td>Antwerp</td></tr>
<tr><td>1924</td><td>Paris</td></tr>
<tr><td>1928</td><td>Amsterdam</td></tr>
<tr><td>1932</td><td>Los Angeles</td></tr>
<tr><td>1936</td><td>Berlin</td></tr>
</table>
<p>
Eleven? It is true that there are ten Games, but it is also true that
the list has eleven elements. Is something wrong here? No. Games had
been scheduled in Berlin in 1916 but were canceled because of the
war. Conceptually, the Games of 1916 are a <em>missing value</em>, and
there are many real world phenomena modeled with time series where it
is common to have some values missing. A system dealing with time
series must be capable of dealing with such cases gracefully and in a
useful way. You don't want your software to return the nine first
Games when you asked for ten, or to crash on the Games of 1916. Or you
don't want your portfolio evaluation software to give up when a quote
is missing. And as a software developer, you don't want to invent ad-hoc
solutions all the time.
<p>
Detecting missing values and handling them intelligently is the
<strong>second important design goal</strong> of the Time Series Framework.
<p>
<hr>
<h3>The pieces of the puzzle</h3><p>
<strong>Time domain</strong>
<p>
<li>
A time domain identifies a <strong>point in time</strong> with an offset from a
base time. The offset is a long integer (64 bit signed integer).
<li>
The <strong>base time</strong> is January 1 0000, corresponding to the index
value 0.
<li>
A <strong>pseudo Gregorian calendar</strong> with the usual definition of leap
years is used. It is not the true thing because it applies from year 0, and
dates before the Gregorian cutover of October 15 1582 do not correspond to
historical dates. Years can be as a large as the size of the numbers used in
the implementation allow.
<li>
The <strong>resolution</strong> of the time series defines the time unit
corresponding to an offset increment. Available units are microsecond, second,
minute, hour, day, month, year. Week is omitted by design.
<li>
The <strong>origin</strong> is a non-negative number of units defining an
offset from the base time. It plays an important role in the time patterns
introduced below. It can also be used to avoid using large integers for indexes
in applications where the range size of a series always fits within a standard
integer. This is especially useful if the programming language does not
support long integers for indexing arrays, like 64 bit Java.
<li>
The <strong>time pattern</strong> defines the time points for which elements
exist. One such pattern is a repeating cycle. A weekly series has a resolution
of day units with a cycle of one day with an element, followed by six days
without elements. A workweek series has day units with a cycle of five days
with elements, followed by two days without elements. The Olympics have year
units with a four-year cycle of one year with games, followed by three years
without. By default, a series has no cycle, meaning elements exist for all
times. The cycle starts with the origin of the time domain. It is possible
for two series with the same resolution and cycle, but not the same origin,
to have no time point in common. The summer and winter Olympics are such a
case.
<p>
<strong>Time index</strong>
<p>
A time index is in a time domain and carries a discrete offset which defines
a point in time. Two time indexes in the same time domain can be compared,
with a larger index corresponding to a later point in time. Adjacent points
in time are represented by adjacent offsets.
<p>
Because time indexes are expected to be used as keys it is important to
implement them as immutable objects.
<p>
<strong>Time range</strong>
<p>
A time range is a pair of time indexes in the same time domain, called
the lower and upper bound. If the lower bound is larger than the upper bound
the range is said to be empty.
<p>
<strong>Observation, value type, missing value</strong>
<p>
An observation has a time index and a value of some type. The value type
must allow the definition of a special value representing missing values,
without restricting the set of useful values; null (nil) can only be used
as this special value if it has no other meaning in the relevant context.
<p>
<strong>Time series</strong>
<p>
A time series maps a set of time indexes to values. All time indexes are
in the same time domain and all values are of the same type. For this
reason we talk of the time domain and the value type of a time series.
<p>
A time series can be defined alternatively as a set of observations,
with each element of the set uniquely identified by its time index.
<p>
A time series has a range defined by the smallest and largest time indexes
of the included observations.
<p>
A time series maintains the abstraction of missing values, which correspond
to time indexes in the range for which no observation exist. This definition
implies that a time series can never have missing values at its boundaries.
This is also true for a subset of a time series, because it is also a
time series.
<p>
<hr>
<h3>Time series storage</h3>
Although an implementation detail, the storage of time series presents an
interesting design question. That a series maps time indexes to values does
not mean that it has to be stored in a dictionary keyed by time indexes. The
<strong>Flyweight</strong> pattern provides a better solution, making use of
the fact that for a given time domain, time indexes can be represented by
integer values, the offsets.
<p>
Two cases must be considered, depending on the frequency of missing
values. In a <strong>regular</strong> time series, missing values are
exceptional. In a <strong>sparse</strong> time series, they are the rule.
<p>
Sparse series are useful for managing irregular events. They are typically
implemented as dictionaries. Missing values are not stored.
Instead of time indexes, only offsets are used as keys. The time domain
is stored only once. Time indexes are reconstructed if and when needed.
<p>
The storage of regular series is straightforward and efficient.
All values, missing or not, are stored into an array. With missing values the
exception, the overhead is reasonable, and because they are self-signaling,
missing values are always detectable. In addition to the array, the series
stores also the time domain and the offset of the time index of the value in
the first array element. A time index can be reconstructed from the time domain
and the sum of the stored offset and the array index of the value.
<p>
<small>
<a rel="license" href="http://creativecommons.org/licenses/by/3.0/">
<img alt="Creative Commons License" style="border-width:0" src="http://i.creativecommons.org/l/by/3.0/80x15.png" /></a><br />
<span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/StillImage" property="dct:title"
rel="dct:type">Time Series Framework Design</span>
by <a xmlns:cc="http://creativecommons.org/ns#" href="http://agent.ch/articles/TimeSeriesDesign.html"
property="cc:attributionName" rel="cc:attributionURL">Jean-Paul Vetterli</a> is licensed under a <a rel="license"
href="http://creativecommons.org/licenses/by/3.0/">Creative Commons Attribution 3.0 Unported License</a>.
</small>Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-59213408054649348572012-07-06T03:47:00.000-07:002012-07-13T10:16:46.627-07:00Walking through the ChronoDB demo (2/2)<hr>
<b>
Important notice. On July 13, 2012, the ChronoDB project was renamed CrNiCKL, which is pronounced like "chronicle". All packages, demos included, have been renamed. The new project website is at <a href="http://agent.ch/timeseries/crnickl/">http://agent.ch/timeseries/crnickl/</a>. The old project remains accessible for a while
at <a href="http://agent.ch/timeseries/chronodb/">http://agent.ch/timeseries/chronodb/</a>. The remainder of this article remains valid <em>mutatis mutandis</em>.
</b>
<hr>
<p>
This post is the second of a two-part commentary on the ChronoDB demo.
In the <a href="http://timeseriessoftware.blogspot.ch/2012/07/walking-through-chronodb-demo-12.html">first part</a> I explained the steps for setting up a ChronoDB database before it can be used to perform useful work.
<p>
Setting up the schema for the demo happens on the last line of our code snippet:
<small><xmp style="background: white">StocksAndForexDemo demo = new StocksAndForexDemo(args[0]);
demo.setUpHyperSQLDatabase();
demo.setUpSchema();
</xmp></small>
Let's focus on this method.
<small><xmp style="background: white">public void setUpSchema() throws Exception {
StocksAndForexSchema schema = new StocksAndForexSchema(db);
schema.createSchema();
// commit all changes
db.commit();
}
</xmp></small>
Whenever the symbol <code>db</code> appears in this post, it stands for the ChronoDB database.
<small><xmp style="background: white">ch.agent.chronodb.api.Database db;
</xmp></small>
The demo needs numerical series and textual attributes. So we need to create two value types. This is done in the <code>StocksAndForexSchema</code> constructor:
<small><xmp style="background: white">db.createValueType("text", false, ValueType.StandardValueType.TEXT.name())
.applyUpdates();
db.createValueType("numeric", false, ValueType.StandardValueType.NUMBER.name())
.applyUpdates();
</xmp></small>
Here we use <b>built-in support</b> for numbers and texts provided by ChronoDB. In other cases we will provide a custom <code>ValueScanner</code>. But the constructor is not finished with its work. It needs to tell ChronoDB that we are going to have numeric series:
<small><xmp style="background: white">UpdatableValueType<ValueType> uvtvt = db.getTypeBuiltInProperty()
.getValueType().typeCheck(ValueType.class).edit();
uvtvt.addValue(uvtvt.getScanner().scan("numeric"), null);
uvtvt.applyUpdates();
</xmp></small>
The method invocation <code>applyUpdates()</code> seen here and there consolidates all pending modifications to an object but <b>does not commit</b> them to permanent storage.
<p>
When the constructor is done, things get more specific, as can be seen from the code of <code>createSchema</code>:
<small><xmp style="background: white">public void createSchema() throws T2DBException {
createCurrencyValueTypeAndProperty();
createSeriesUnitValueTypeAndProperty();
createTickerProperty();
createStocksSchema();
createExchangeRatesSchema();
createTopLevelChronicles();
}
</xmp></small>
In the demo we have a class for currencies and we want to use a <b>custom value scanner</b>:
<small><xmp style="background: white">UpdatableValueType<Currency> uvt =
db.createValueType("Currency", true,
"ch.agent.chronodb.demo.CurrencyValueScanner");
</xmp></small>
The Currency object in this demo is not very useful. In a real world application it would have
more responsibilities, like computing exchange rates. Its purpose here is only to show how we
set up ChronoDB to use problem-related classes. Not shown here is how to add currency values and how to create the currency property, as it's straightforward. We also skip the details of creating other value types and properties and turn to the creation of a schema for stocks.
<p>
We want to represent a stock using a chronicle with two attributes, ticker, and currency, and with three series, price, volume, and splits. Price and volume have a custom series attribute, unit. The first thing to do is to create a schema.
The demo does not use the possibility to <b>inherit from another schema</b>.
<small><xmp style="background: white">UpdatableSchema schema = db.createSchema("Stocks", null);
</xmp></small>
Attributes are then added to the schema. It is necessary to provide numbers for
attributes and series. At first sight one would ask why are these numbers not hidden by the software. The answer is schema inheritance and the possibility to not only remove and modify attributes and series but also to insert them in precise positions.
<small><xmp style="background: white">schema.addAttribute(1);
schema.setAttributeProperty(1, db.getProperty("Ticker", true));
schema.addAttribute(2);
schema.setAttributeProperty(2, db.getProperty("Currency", true));
</xmp></small>
The following piece of code defines the price series as a workweek numeric series, with a currency unit:
<small><xmp style="background: white">schema.addSeries(1);
schema.setSeriesName(1, "price");
schema.setSeriesDescription(1, "close price");
schema.setSeriesType(1, db.getValueType("numeric"));
schema.setSeriesTimeDomain(1, Workday.DOMAIN);
schema.addAttribute(1, 5);
schema.setAttributeProperty(1, 5, db.getProperty("Unit", true));
schema.setAttributeDefault(1, 5, "currency");
</xmp></small>
By default a series does use sparse time series. The automatic use of sparse time series can be configured in the schema. This is done for splits series in the demo.
Even if not configured, applications still have the possibility to force sparsity when getting data.
<p>
The last step in setting up the schema is to create top level chronicles. The demo uses two collections: stocks and exchange rates. The code below shows how to create the top chronicle hosting the exchange rate collection.
<small><xmp style="background: white">Schema forexSchema = db.getSchemas("Forex").iterator().next();
UpdatableChronicle forex = db.getTopChronicle().edit()
.createChronicle("forex", false, "Exchange rate data", null, forexSchema);
forex.applyUpdates();
</xmp></small>
To create a chronicle you need to have a parent chronicle. For a top-level chronicle, the parent is <u>the</u> top chronicle, which is virtual and which is named after the database, "demo" in this case.
<p>
With top-level chronicles created, the demo can go ahead.
Once the required schemas have been set up, application rarely, if ever, need to do anything about them. Millions of chronicles can be created, and their attributes and series are automatically available without dong anything, except setting specific values.
In many cases it is not even necessary to set values of attributes. Take the case of american stocks. It is a simple matter to define a schema for them, inheriting from
the "Stocks" schema we made in the demo:
<small><xmp style="background: white">UpdatableSchema schema = db.createSchema("American stocks", "Stocks");
Property<Currency> currency = db.getProperty("Currency", true)
.typeCheck(Currency.class);
schema.setAttributeDefault(2, currency.scan("USD"));
</xmp></small>
With this new schema, you can now create a chronicle collection for american stocks (perhaps a nested collection of the stocks collection), and all members will automatically be in USD.
<p>
As a final remark, it is important to note that as the default value of a chronicle attribute can always be overriden (it's named a default value after all), the value of a series attribute cannot. It keeps its default value, as defined in the schema. If you say in the schema that the unit of a series foo is bar then in all collections having this schema, the foo series unit will be bar. The same goes for built-in attributes: name, type, time domain, and sparsity of a series cannot be changed once defined. This does not restrict the modeling freedom. As the demo as shown you can have price series in different currencies. The price and volume have a unit ("in currency", and "in number of shares"), but the price series unit "in currency" simply tells to look at the chronicle's currency. So you will have your <a href="http://www.bloomberg.com/quote/7203:JP">Toyotas</a> in quoted in yens and your <a href="http://www.bloomberg.com/quote/RNO:FP">Renaults</a> in euros.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-67933795963005360052012-07-05T12:32:00.000-07:002012-07-13T10:17:44.315-07:00Walking through the ChronoDB demo (1/2)<hr>
<b>
Important notice. On July 13, 2012, the ChronoDB project was renamed CrNiCKL, which is pronounced like "chronicle". All packages, demos included, have been renamed. The new project website is at <a href="http://agent.ch/timeseries/crnickl/">http://agent.ch/timeseries/crnickl/</a>. The old project remains accessible for a while
at <a href="http://agent.ch/timeseries/chronodb/">http://agent.ch/timeseries/chronodb/</a>. The remainder of this article remains valid <em>mutatis mutandis</em>.
</b>
<hr>
<p>
A demo package is available for downloading from the <a href="http://agent.ch/timeseries/chronodb">ChronoDB project</a> website or from <a href="http://sourceforge.net/projects/chronodb/files/chronodb-demo-1.0.0.jar/download">SourceForge</a>. In a short series of posts I will comment on a few important details. I hope these explanations will be helpful.
<p>
Explanations will focus on the following code snippet from the static main method of <code>StocksAndForexDemo</code>:
<small><xmp style="background: white">// args[0] is name of parameter file with key-value pairs ...
StocksAndForexDemo demo = new StocksAndForexDemo(args[0]);
demo.setUpHyperSQLDatabase();
demo.setUpSchema();
// etc.
</xmp></small>
<p>
The constructor invocation <strong><code>new StocksAndForexDemo(args[0])</code></strong>
initializes a ChronoDB database, kept in a private member inside the demo:
<small><xmp style="background: white">private ch.agent.chronodb.api.Database db;
</xmp></small>
The actual work is done by <code>ch.agent.chronodb.api.SimpleDatabaseManager</code>, which is one of the few non-interface classes in that package. It is provided to make it easier to write test cases (and demos). It sets up the database using parameters provided in a file on the file system or the class path. Here is an extract from such a file:
<small><xmp style="background: white">db.name=demo
db.class=ch.agent.chronodb.jdbc.JDBCDatabase
session.jdbcDriver=org.hsqldb.jdbc.JDBCDriver
session.jdbcUrl=jdbc:hsqldb:mem:demodb
session.db=
session.user=sa
session.password=
# etc.
</xmp></small>
Parameters with names beginning with "db." are the most important: <em>db.name</em> names the database and must be unique within a running system; <em>db.class</em> names the implementation class. Although it is in principle possible to run multiple databases simultaneously, <code>SimpleDatabaseManager</code> currently supports only one database. Parameters beginning with "session." are specific to the JDBC implementation. Implementations more sophisticated than <code>JDBCDatabase</code> used in this simple demo will have more parameters, some of which are named in the interface <code>ch.agent.chronodb.impl.DatabaseBackend</code>.
<p>
At this point, a ChronoDB database object and a JDBC connection are ready for use but there is absolutely nothing in the database yet. In fact, the demo uses an in-memory <a href="http://hsqldb.org/">HyperSQL</a> database. Before the demo starts and after the demo terminates, the database does not exist. So the next step is to create the tables and indexes expected by the JDBC implementation of ChronoDB.
<p>
This is done by the method invocation <strong><code>demo.setUpHyperSQLDatabase()</code></strong> which sends SQL data definition language (DDL) statements to the database engine for execution. The DDL is taken from <strong><code>Resources/HyperSQL_DDL_base.sql</code></strong>. This file is in
<strong><code>chronodb-jdbc-1.0.0.jar</code></strong> and therefore on the class path. The DDL defines all tables required by the base system, with various indexes and constraints to enforce referential integrity. Browse the SQL file if you need details. Noteworthy are the few non DDL statements at the end of the file, which initialize the database with the built-in properties needed when defining a series in a schema. These properties require in turn the corresponding built-in value types.
<p>
These value types are:
<ul>
<li><em>name</em>, a string type enforcing a minimalist naming policy
<li><em>type</em>, for values defining the type of a series
<li><em>timedomain</em>, a restricted type for time domains, with some predefined values: daily, datetime, monthly, workweek, and yearly
<li><em>binary</em>, a boolean type
</ul>
The properties, with their value type in parentheses, are:
<ul>
<li>Symbol (name)
<li>Type (type)
<li>Calendar (timedomain)
<li>Sparsity (binary)
</ul>
These value types and properties could in theory be provided virtually by the software, but
in a JDBC implementation they are physically required because of referential integrity.
For more information on time domains, please consult the documentation of the <a href="http://agent.ch/timeseries/t2/">Time2 Library project</a>. For more information on the other properties please consult the documentation of the <a href="http://agent.ch/timeseries/chronodb/">ChronoDB database project</a>.
<p>
At this point the database is ready for useful work directly related to the problem at hand: setting up the schema for the demo. This will be the subject of a forthcoming post.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-48497126024461495382012-06-27T13:37:00.001-07:002012-06-27T13:38:25.468-07:00The ChronoDB database for time series releasedVersion
<a href="http://agent.ch/timeseries/chronodb/RELEASE_NOTES_v1.html#v1_0_0">1.0.0</a> of the ChronoDB database was released on June 27, 2012. It is available from <a href="http://sourceforge.net/projects/chronodb/">http://sourceforge.net/projects/chronodb/</a> and <a href="http://agent.ch/timeseries/chronodb/">http://agent.ch/timeseries/chronodb/</a>.
<p/>
ChronoDB is a data manager for generic time series written in Java. It supports time series of any type. A flexible schema subsystem allows to organize simply but powerfully very large data sets. The software consists of an API and a generic implementation layer running on top of an SQL or a NoSQL system.
<p/>
A demo is also available from the same sites. A future post on this blog will explain all about ChronoDB.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-28573703313313929952012-03-21T14:56:00.002-07:002012-03-22T01:46:25.175-07:00Maintenance release of the Time2 LibraryVersion <a href="http://agent.ch/articles/RELEASE_NOTES_v1.html#v1_1_4">1.1.4</a> of the Time2 Library was released on March 21, 2012 at <a href="http://time2.sourceforge.net/">http://time2.sourceforge.net/</a> and <a href="http://agent.ch/timeseries/t2/">http://agent.ch/timeseries/t2/</a>. <br />
<p>This maintenance release is plug-compatible with the previous version of the software. The internal management of diagnostic messages and exceptions has been improved in three ways:<br />
<ol><li>Diagnostic messages are fetched and formatted only when actually needed. This improves performance, especially in the case where not all messages are logged by the application environment.<br />
<li>The library has now its own exception type, T2Exception. Because it is a subclass of the exception type used previously there is no compatibility issue.<br />
<li>Messages are now keyed symbolically instead of literally. This provides many benefits to the programmer. One of these is readily visible in the Eclipse IDE where the text of the diagnostic message is displayed as a tooltip when the mouse pointer idles over a message key. The screenshot below shows this in action.<br />
</ol><p><a href="http://agent.ch/images/Eclipse-message-tooltip-larger.png"><img src="http://agent.ch/images/Eclipse-message-tooltip.png" /></a>. <p>As an aside, the snapshot shows a piece of JUnit testing code. Writing software is easier and faster with <a href="http://www.extremeprogramming.org/rules/testfirst.html">test-first development</a>. JUnit is a simple and powerful testing pattern for Java. We can thank <a href="http://junit.sourceforge.net/doc/cookbook/cookbook.htm">Beck and Gamma</a> for it. Note that the original idea was developed for Smalltalk by <a href="http://www.xprogramming.com/testfram.htm">Kent Beck</a> in 1989 and thus predates Java. <p><b>Coming soon</b> <p>A data management system for time series running on top of the Time2 Library is in the pipeline. I hope to release it one of these weeks...Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-45432753715137651512011-11-30T10:53:00.001-08:002012-03-22T01:47:47.911-07:00Version 1.1.3 (stable) of the Time2 Library brings a new utility<p>A new version of the Time2 Library is available at <a href="http://time2.sourceforge.net/">http://time2.sourceforge.net/</a> and <a href="http://agent.ch/timeseries/t2/">http://agent.ch/timeseries/t2/</a>. <br />
<p>This new release comes with a small utility for scanning dates and times in arbitrary formats. Here is a quick usage example:<br />
<p><blockquote style="font-size: smaller"><pre style="font-size: smaller">String pattern =
"\\S+ (\\S+) (\\d+) (\\d\\d):(\\d\\d):(\\d\\d) \S+ (\\d\\d\\d\\d)";
int[] groups = {6, 1, 2, 3, 4, 5};
String[] months =
{"Jan", "Feb", "Mar", "Apr", "May", "Jun",
"Jul", "Aug", "Sep", "Oct", "Nov", "Dec"};
DateTimeScanner scanner = new DateTimeScanner(pattern, groups);
scanner.setMonths(months);
scanner.setDomain(DateTime.DOMAIN);
TimeIndex time = scanner.scan("Wed Nov 30 12:29:23 CET 2010");
assertEquals("2010-11-30 12:29:23", time.toString());
</pre></blockquote>Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-58563672893975572462011-10-10T06:20:00.000-07:002011-10-10T06:23:01.316-07:00Version 1.1.2 (stable) of the Time2 Library supports ISO 8601A new version of the Time2 Library is available at <a href="http://time2.sourceforge.net/">http://time2.sourceforge.net/</a> and <a href="http://agent.ch/timeseries/t2/">http://agent.ch/timeseries/t2/</a>. <br />
<br />
This new release brings improved support for the representation of dates and times as defined by the ISO 8601:2004 standard. Basic and extended date and time strings are allowed. In the extended format, the current time, as I'm writing, is represented as 2011-10-10T12:42:53Z. The Z indicates UTC time. The same time in the basic format is written 20111010T124253Z. Because time zones and daylight savings time are irrelevant in the Time2 Library, the Z letter can be omitted.<br />
<br />
When using the extended format, the T separating date and time can be replaced with a blank. So the example can also be represented as 2011-10-10 12:42:53, arguably nicer on the eye. This more natural format is used by Time2.toString().<br />
<br />
Local times can be specified using a time zone offset. For example, I am writing this from Switzerland, where the time zone is CEST, 2 hours ahead of UTC. In the ISO 8601:2004 standard, this can be written 2011-10-10T14:42:53-02.<br />
<br />
There are some exotic features in the ISO 8601 standard. Some of them, like leap seconds or the use of hour 24 for midnight, are supported. Others, like week dates and ordinal dates are not.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-40162857874202013972011-10-03T09:07:00.001-07:002012-03-22T01:48:58.146-07:00Why make it complicated when you can make it impossible?Dates and times may be quite familiar but when you need to do something with them, especially in software, you are confronted with a glorious mess. To get an idea, spend a few moments on the subject in Wikipedia, Google, or the Java documentation. Here are some of the things you will find.<br />
<br />
<b>There is no year zero, except when there is one...</b><br />
<br />
If you are a historian, there is no year 0. The year AD 1 immediately follows the year 1 BC (see <a href="http://en.wikipedia.org/wiki/Anno_Domini">Anno Domini</a> in Wikipedia). On the other hand, if you are an astronomer, there is a year 0 (see <a href="http://en.wikipedia.org/wiki/Astronomical_year_numbering">Astronomical year numbering</a> in Wikipedia). In the <a href="http://en.wikipedia.org/wiki/ISO_8601">ISO 8601</a> international standard for representing dates and times, years are represented by four digits from 0000 to 9999. So, if you are following the standard, there is a year 0. But with XML, there is a problem. XML time was inspired by an early version of ISO 8601 and disallows year 0. Fortunately, this is likely to change in the future to agree with more recent versions of ISO 8601. See <a href="http://www.w3.org/TR/xmlschema-2/#dateTime">XML Schema Part 2: Datatypes Second Edition</a> from W3C.<br />
<br />
<b>If you have an ancestor born on October 10, 1582, stick to the ISO standard!</b><br />
<br />
In some countries, October 15, 1582 immediately followed October 4 (see <a href="http://en.wikipedia.org/wiki/Gregorian_calendar">Gregorian calendar</a> in Wikipedia). The ISO 8601 international standard states that every date must be consecutive and that dates before October 15 1582 can be used by mutual agreement when exchanging information. So, under the standard, there would have been no hole between October 4 and 15, 1582, and your ancestor would have been born on an existing day.<br />
<br />
<b>Some minutes have 61 seconds</b><br />
<br />
Nowadays, time is measured with atomic clocks (see <a href="http://en.wikipedia.org/wiki/UTC">Coordinated Universal Time</a> (UTC) in Wikipedia). Atomic time does not exactly match the earth's rotation and a so-called leap second can be added or substracted from UTC at the end of June or December to avoid the discrepancy growing beyond a second. So, minutes have 59, 60, or 61 seconds. For more information on leap seconds, consult <a href="http://www.ietf.org/rfc/rfc3339.txt">Date and Time on the Internet: Timestamps (RFC3339)</a> at the IETF, and <a href="http://tycho.usno.navy.mil/leapsec.html">Leap Seconds</a> at the US Naval Observatory. Note because it is not known long in advance when one will be introduced, leap seconds cannot be programmed in software.<br />
<br />
<b>Avoid the Dutch calendar between 1909 and 1937</b><br />
<br />
When using local time, time zones and daylight savings time are issues to consider. The fact that most methods of the Java <a href="http://download.oracle.com/javase/1.4.2/docs/api/java/util/Date.html">Date class</a> are deprecated shows that getting such things right is not straightforward. <a href="http://download.oracle.com/javase/1.4.2/docs/api/java/util/Calendar.html">Calendar</a> and <a href="http://joda-time.sourceforge.net/">Joda Time</a> are better, but their complexity can be seen as further proof that dates and times are messy. The ISO 8601 standard attempts to sidestep the whole question by not supporting time zones at all. Time is either local or expressed as an offset from UTC. But even then, there is a problem. The offset used by the ISO standard is in hours and minutes. But there are known cases where this is not sufficient. Here is a quote from <a href="http://www.ietf.org/rfc/rfc3339.txt">RFC 3339</a>:<br />
<br />
<blockquote>1937-01-01T12:00:27.87+00:20<br />
<br />
This represents the same instant of time as noon, January 1, 1937,<br />
Netherlands time. Standard time in the Netherlands was exactly 19<br />
minutes and 32.13 seconds ahead of UTC by law from 1909-05-01 through<br />
1937-06-30. This time zone cannot be represented exactly using the<br />
HH:MM format, and this timestamp uses the closest representable UTC<br />
offset.<br />
</blockquote><br />
<b>The Y2.038K problem</b><br />
<br />
If you thought the year 2000 problem was interesting, then you will like 2038, when the Unix clock will wrap around. Such problems and many facts about dates and times are documented in Gilbert Healton's <a href="http://www.exit109.com/~ghealton/y2k/yrexamples.html">The Best of Dates, The Worst Of Dates</a>.<br />
<br />
<b>Time and the Time2 Library</b><br />
<br />
When I wrote the <a href="http://agent.ch/timeseries/t2/">Time2 Library</a>, I was looking for something lightweight to represent dates and the time of day. Dates and times are only a small part of what the library does and I did not want to spend too much time on time. But I was not happy with Java's Date because it is mostly deprecated. Other possible solutions were not exactly lightweight. So I decided to write my own solution and to keep it simple, with the requirement to implement part of the ISO 8601 specification (ignoring durations and intervals): all years between 0000 and 9999 valid, no hole in October 1582, no support for time zones and daylight savings time, but tolerance for time zone offsets and leap seconds (the last two things are still on the to-do list.)Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-30515988201361877982011-09-21T06:45:00.000-07:002011-09-21T06:49:04.826-07:00Version 1.1.1 (stable) of the Time2 Library releasedA new version of the Time2 Library is available at <a href="http://time2.sourceforge.net/">http://time2.sourceforge.net/</a> and <a href="http://agent.ch/timeseries/t2/">http://agent.ch/timeseries/t2/</a>. <br />
<br />
This release brings <b>improved time comparisons</b>. Previously it was not possible to compare times from different domains. Now, when domain differs, times are converted automatically.<br />
<br />
When resolutions differ, the time with the lowest resolution is converted to the one with the highest resolution. For example comparing a <em>Day</em> to a <em>Year</em> will compare the day to the implicit default day of the year (which is January 1).<br />
<br />
When resolutions do not differ, the times are converted to the unrestricted time domain for their resolution. For example a <em>Workday</em> and a <em>ThirdFriday</em> are both converted to a <em>Day</em> before comparing.<br />
<br />
These semantics are not meaningful for all possible time domains. Users of "exotic" time domains should consider writing a subclass of <em>Time2</em> and override the <em>compareTo</em> method.Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-70914228230614460362011-09-01T16:19:00.000-07:002011-09-01T16:19:07.484-07:00StockChart: combining the Time2 Library with JFreeChartA demo package for the <a href="http://agent.ch/timeseries/t2/">Time2 library</a></a> was released on September 1. It includes <b>StockChart</b>, a program to create price and volume charts for stocks. For this it relies heavily on <a href="http://www.jfree.org/jfreechart/">JFreeChart</a>, a flexible and powerful Java library for drawing charts. Charts can be saved as PNG images or as vector graphics in SVG files, thanks to <a href="http://xmlgraphics.apache.org/batik/">Apache Batik</a>.<br />
<br />
The demo uses example files from <b>Foo & Bar, Inc</b> (trading symbol FBI), a company invented for the occasion. Here is a PNG chart produced by the demo:<br />
<br />
<a href="http://agent.ch/images/FBI-1991Q1.svg"><img src="http://agent.ch/images/FBI-1991Q1.png" /></a>.<br />
<br />
Clicking on the image shows the SVG version of the chart. If your browser supports SVG (like Firefox) you will be able to scale the image by resizing the window without reducing the picture quality.<br />
<br />
The demo can be downloaded from <a href="https://sourceforge.net/projects/time2/files/">SourceForge</a> or from the <a href="http://agent.ch/timeseries/t2/">project website</a>.<br />
<br />
As noted in the chart title, FBI quotes have not been adjusted for stock splits. <br />
The halving of the price on March 18 looks suspiciously like a 2:1 ("two for one") split. A future version of the demo will take care of this problem. Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-20607217803408942542011-08-22T09:08:00.000-07:002011-08-22T11:07:57.453-07:00Version 1.1.0 (stable) of the Time2 Library releasedA new version of the Time2 Library is available at <a href="http://time2.sourceforge.net/">http://time2.sourceforge.net/</a> and <a href="http://agent.ch/timeseries/t2/">http://agent.ch/timeseries/t2/</a>. I have clarified the rule on nulls. In one sentence, it is illegal to put a null into a time series unless null was defined as the representative of missing values for the type. The rule was already present in previous versions, but it is now better documented, and also covers <em>Filler</em> callbacks.<br />
<br />
Unlike announced in my previous post, the new version number is 1.1.0 and not 1.0.1. Increasing the medium version number signals the presence of incompatible modifications, in this case the addition of a throws clause to two (rarely used) methods of <em>TimeIndexable</em>. Details are available in the release notes included in the software.<br />
<br />
With this release, the software leaves <b>beta</b> status and is now <b>stable</b>.<br />
<br />
Coming soon is a little demo of time series with graphics!Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-10107994556648357062011-08-17T16:46:00.000-07:002011-08-17T16:46:25.740-07:00Version 1.0.0 (beta) of Time2 Library releasedA slightly modified version of the Time2 Library is available at <a href="http://time2.sourceforge.net/">http://time2.sourceforge.net/</a> and <a href="http://agent.ch/timeseries/t2/">http://agent.ch/timeseries/t2/</a>. I have added a README file and release notes. The names of distribution files now include version numbers.<br />
<br />
Version 1.0.1 is expected for next week. It will come without the "beta" qualifier. Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0tag:blogger.com,1999:blog-3614203688951258463.post-605349178019661052011-07-25T15:34:00.000-07:002011-07-25T15:37:02.668-07:00Welcome to the Time Series Software blog !<span style="font-size: small;"><span style="font-family: "Helvetica Neue",Arial,Helvetica,sans-serif;">This blog is for notes and short articles about the <a href="http://time2.sourceforge.net/">Time2 Library</a>, a piece of Java software I wrote and released ten days ago as an open source project. There will also be more general material related to time series software and applications. </span></span>Jean-Paul Vetterlihttp://www.blogger.com/profile/08843571056205135064noreply@blogger.com0