<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Electric Words &#187; postgresql</title>
	<atom:link href="http://www.electricwords.org/tags/postgresql/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.electricwords.org</link>
	<description>Dan Ellis's technical blog</description>
	<lastBuildDate>Wed, 04 May 2011 19:35:40 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.9.2</generator>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
			<item>
		<title>Fuzzy date matching in PostgreSQL</title>
		<link>http://www.electricwords.org/2008/11/fuzzy-date-matching-in-postgresql/</link>
		<comments>http://www.electricwords.org/2008/11/fuzzy-date-matching-in-postgresql/#comments</comments>
		<pubDate>Wed, 05 Nov 2008 00:45:21 +0000</pubDate>
		<dc:creator>Dan Ellis</dc:creator>
				<category><![CDATA[Databases]]></category>
		<category><![CDATA[postgresql]]></category>
		<category><![CDATA[Web development]]></category>

		<guid isPermaLink="false">http://www.electricwords.org/?p=34</guid>
		<description><![CDATA[As part of one of my side-projects, I wanted a way for users of a web site to search for events by date, but with some flexibility. I also wanted users who are creating events to be able to express uncertainty about the dates on which they happened.

For example, if I record an event that [...]]]></description>
			<content:encoded><![CDATA[<p>As part of one of my side-projects, I wanted a way for users of a web site to search for events by date, but with some flexibility. I also wanted users who are creating events to be able to express uncertainty about the dates on which they happened.</p>

<p>For example, if I record an event that happened sometime in September 1998, I can say it happened on September 15th, 1998 +/- 15 days. That should then be included in the results of a query for events that happened on October 1st, 1998 +/ 1 month, or June 1st, 1998 +/- 6 months, or even September 10th, 1998 +/- 1 day. An exact date would be represented as +/- 0 days.</p>

<p>One way to visualise these is as line segments on a literal date line, stretching from the past into the future, with a mark per day. PostgreSQL&#8217;s spatial types can be used to represent this in an indexable way. First of all, we&#8217;ll create a composite type to represent fuzzy dates:</p>


<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">CREATE</span> TYPE fuzzydate <span style="color: #993333; font-weight: bold;">AS</span> <span style="color: #66cc66;">&#40;</span>midpoint date<span style="color: #66cc66;">,</span> fuzziness interval<span style="color: #66cc66;">&#41;</span>;</pre></div></div>


<p>The midpoint and fuzziness fields should be self-explanatory. Next, we create a function that will take a fuzzydate and convert it into a box. (PostgreSQL doesn&#8217;t have an &#8216;overlaps&#8217; operator for line segments, so I chose to use zero-height boxes.) The function calculates the start and end dates of the date range, and uses <tt>extract</tt> to convert them into epoch times (the number of seconds since the beginning of 1970). These are then divided by 86400 to get the number of days since 1970. These values form the x coordinates of the returned box.</p>


<div class="wp_syntax"><div class="code"><pre class="plsql" style="font-family:monospace;"><span style="color: #00F;">CREATE</span> <span style="color: #00F;">FUNCTION</span> GeometricDate<span style="color: #00F;">&#40;</span>fd fuzzydate<span style="color: #00F;">&#41;</span> RETURNS box <span style="color: #00F;">AS</span> $$
  <span style="color: #00F;">DECLARE</span>
    start_day <span style="color: #00F;">INTEGER</span><span style="color: #00F;">;</span>
    end_day <span style="color: #00F;">INTEGER</span><span style="color: #00F;">;</span>
  <span style="color: #00F;">BEGIN</span>
    start_day <span style="color: #00F;">:=</span> <span style="color: #000;">EXTRACT</span><span style="color: #00F;">&#40;</span>epoch <span style="color: #00F;">FROM</span> fd<span style="color: #00F;">.</span>midpoint <span style="color: #00F;">-</span> fd<span style="color: #00F;">.</span>fuzziness<span style="color: #00F;">&#41;</span><span style="color: #00F;">::</span><span style="color: #00F;">INTEGER</span> <span style="color: #00F;">/</span> <span style="color: #800;">86400</span><span style="color: #00F;">;</span>
    end_day <span style="color: #00F;">:=</span> <span style="color: #000;">EXTRACT</span><span style="color: #00F;">&#40;</span>epoch <span style="color: #00F;">FROM</span> fd<span style="color: #00F;">.</span>midpoint <span style="color: #00F;">+</span> fd<span style="color: #00F;">.</span>fuzziness<span style="color: #00F;">&#41;</span><span style="color: #00F;">::</span><span style="color: #00F;">INTEGER</span> <span style="color: #00F;">/</span> <span style="color: #800;">86400</span><span style="color: #00F;">;</span>
    <span style="color: #00F;">RETURN</span> box<span style="color: #00F;">&#40;</span>point<span style="color: #00F;">&#40;</span>start_day<span style="color: #00F;">,</span> <span style="color: #800;">0</span><span style="color: #00F;">&#41;</span><span style="color: #00F;">,</span> point<span style="color: #00F;">&#40;</span>end_day<span style="color: #00F;">,</span> <span style="color: #800;">0</span><span style="color: #00F;">&#41;</span><span style="color: #00F;">&#41;</span><span style="color: #00F;">;</span>
  <span style="color: #00F;">END</span><span style="color: #00F;">;</span>
$$ LANGUAGE <span style="color: #F00;">'plpgsql'</span> IMMUTABLE<span style="color: #00F;">;</span></pre></div></div>


<p>Now we can now perform a query like this:</p>


<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">SELECT</span> GeometricDate<span style="color: #66cc66;">&#40;</span><span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">'2008-11-04'</span><span style="color: #66cc66;">,</span> <span style="color: #ff0000;">'2 days'</span><span style="color: #66cc66;">&#41;</span><span style="color: #66cc66;">&#41;</span>;</pre></div></div>


<p>to get a result like (14189,0),(14185,0), so let&#8217;s create a table that we can perform our date range queries on:</p>


<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">CREATE</span> <span style="color: #993333; font-weight: bold;">TABLE</span> events <span style="color: #66cc66;">&#40;</span>
  id serial <span style="color: #993333; font-weight: bold;">PRIMARY</span> <span style="color: #993333; font-weight: bold;">KEY</span><span style="color: #66cc66;">,</span>
  name varchar<span style="color: #66cc66;">&#40;</span><span style="color: #cc66cc;">100</span><span style="color: #66cc66;">&#41;</span><span style="color: #66cc66;">,</span>
  fdate fuzzydate
<span style="color: #66cc66;">&#41;</span>;</pre></div></div>


<p>Note that the fdate column is only storing the original midpoint and fuzziness. So where is the geometric representation? Well, PostgreSQL allows us to index not just a column, but the results of applying a function to a column:</p>


<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">CREATE</span> <span style="color: #993333; font-weight: bold;">INDEX</span> events_fdate_index <span style="color: #993333; font-weight: bold;">ON</span> events <span style="color: #993333; font-weight: bold;">USING</span> gist <span style="color: #66cc66;">&#40;</span>GeometricDate<span style="color: #66cc66;">&#40;</span>fdate<span style="color: #66cc66;">&#41;</span><span style="color: #66cc66;">&#41;</span>;</pre></div></div>


<p>Now we can use the geometric date in a query like this one, which says &#8220;give me all the events that were within 10 days before or after November 3rd, 2008&#8243;:</p>


<div class="wp_syntax"><div class="code"><pre class="sql" style="font-family:monospace;"><span style="color: #993333; font-weight: bold;">SELECT</span> <span style="color: #66cc66;">*</span> <span style="color: #993333; font-weight: bold;">FROM</span> events <span style="color: #993333; font-weight: bold;">WHERE</span> GeometricDate<span style="color: #66cc66;">&#40;</span>fdate<span style="color: #66cc66;">&#41;</span> &amp;amp;&amp;amp; GeometricDate<span style="color: #66cc66;">&#40;</span><span style="color: #ff0000;">'2008-11-03'</span><span style="color: #66cc66;">,</span> <span style="color: #ff0000;">'10 days'</span><span style="color: #66cc66;">&#41;</span>;</pre></div></div>


<p>The really important thing to note here is that GeometricDate is <em>not</em> being called for every row in the table. The result of GeometricDate is taken directly from the index we created, and it&#8217;s the <tt>IMMUTABLE</tt> flag on the function that tells PostgreSQL that it&#8217;s okay to do this.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.electricwords.org/2008/11/fuzzy-date-matching-in-postgresql/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

