Re: [sup-talk] [PATCH] xapian: do less work for update_message_state

[Olly Betts <olly@survex.com>]

Olly Betts writes:
> On 2009-10-01, Rich Lane <rlane@club.cc.cmu.edu> wrote:
> > Excerpts from William Morgan's message of Thu Oct 01 09:46:20 -0400 2009:
> >> Reformatted excerpts from Rich Lane's message of 2009-09-30:
> >> > They're about 3 times faster on my machine with this patch. An
> >> > optimization the Xapian devs have been planning to make (and that this
> >> > patch is necessary to take advantage of) should increase performance
> >> > much more.
> >> 
> >> Awesome. Out of curiousity, what's the optimization?
> >
> > replace_document currently deletes all the old postings and inserts new
> > ones. It can be optimized to make the minimal set of modifications.
> 
> This is the ticket for it:
> 
> http://trac.xapian.org/ticket/250

This has now been fixed in Xapian SVN trunk, and I've backported the
patches to Xapian's 1.0 branch.  You can find snapshot tarballs including
these changes here:

http://oligarchy.co.uk/xapian/branches/1.0/

Xapian's testsuite passes (including the additional test coverage which I
also backported), and I looked over each change carefully, but I would be
interested to see some real world testing, particularly in the situation
which these changes are intended to improve (i.e. speed of adding/removing
tag terms from existing documents).  I'm not expecting problems, but this
is a larger change than we usually backport, so wider testing would be a
good thing.

If you are using Xapian 1.0's default backend (flint), you'll need this
patch (which was merged before sup 0.10) to get full benefit from the
changes:

http://article.gmane.org/gmane.mail.sup.devel/177

I've uploaded a snapshot of 1.0 with this patch to Debian experimental.
It hasn't built for all Debian architectures yet, but is available for at
least amd64 and x86, which are probably the most popular two.

If you aren't sure how to pull in packages from experimental, see:

http://wiki.debian.org/DebianExperimental

I've also put it in a Launchpad PPA for all currently supported Ubuntu
releases, and it has built for all of them already:

https://launchpad.net/~ojwb/+archive/experimental/

Cheers,
    Olly

_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk