[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[sup-talk] Multiple documents for a single docid in Xapian?

Recently I've gotten into a situation where sup is crashing.  The traceback is
at the end.  The first time I did a little digging, a git pull and finallly
gave up and rebuilt my index.  It happened again this morning and I dug in a
bit more.  (What follows is based on a pull from master as of a few minutes
ago.)

I added a bit of debugging at the point of the crash:  (around line 555 of
index.rb)

  def find_docid id
    docids = term_docids(mkterm(:msgid,id))
    #fail unless docids.size <= 1
    warn "got #{docids.size} (expected 1 or less) for #{id}" unless docids.size <= 1
    docids.first
  end

Basically instead of failing I log a warning and move on.  For some reason I
am seeing 2 entries in the index for that particular docid.  Any idea how this
could happen?  Is there a recommended way to fix this?  My little warning hack
makes me nervous.  I assume I can just delete one of them from the index but I
haven't spent much time understanding how the index is set up and I'm
concerned about unintended consequences...

Interestingly I get the warning 4 times:

[Mon May 16 12:14:26 -0500 2011] WARNING: got 2 (expected 1 or less) for 1305558622-sup-33@arrakis.es.net
[Mon May 16 12:14:26 -0500 2011] WARNING: got 2 (expected 1 or less) for 1305558622-sup-33@arrakis.es.net
[Mon May 16 12:14:29 -0500 2011] WARNING: got 2 (expected 1 or less) for 1305558622-sup-33@arrakis.es.net
[Mon May 16 12:14:29 -0500 2011] WARNING: got 2 (expected 1 or less) for 1305558622-sup-33@arrakis.es.net

I originally got this after an offline IMAP run when it was scanning for new
messages to add to the index.  The message in question is actually in my
~/.sup/sent.mbox file and so it wasn't pulled from IMAP -- I'm guessing it
must have just checked the sent mbox file after it finished checking the
offline IMAP directories.  I'm not really sure.

Any ideas?

Thanks,

Jon


--- RuntimeError from thread: poll after loading inbox

/Users/jdugan/projects/sup/mainline/lib/sup/index.rb:556:in `find_docid'
/Users/jdugan/projects/sup/mainline/lib/sup/index.rb:562:in `find_doc'
/Users/jdugan/projects/sup/mainline/lib/sup/index.rb:572:in `get_entry'
/Users/jdugan/projects/sup/mainline/lib/sup/index.rb:200:in `build_message'
/opt/local/lib/ruby/1.8/monitor.rb:242:in `synchronize'
/Users/jdugan/projects/sup/mainline/lib/sup/index.rb:581:in `synchronize'
/Users/jdugan/projects/sup/mainline/lib/sup/index.rb:200:in `build_message'
/Users/jdugan/projects/sup/mainline/lib/sup/util.rb:618:in `send'
/Users/jdugan/projects/sup/mainline/lib/sup/util.rb:618:in `method_missing'
/Users/jdugan/projects/sup/mainline/lib/sup/poll.rb:159:in `poll_from'
/Users/jdugan/projects/sup/mainline/lib/sup/maildir.rb:106:in `poll'
/Users/jdugan/projects/sup/mainline/lib/sup/util.rb:615:in `each_with_index'
/Users/jdugan/projects/sup/mainline/lib/sup/maildir.rb:105:in `each'
/Users/jdugan/projects/sup/mainline/lib/sup/maildir.rb:105:in `each_with_index'
/Users/jdugan/projects/sup/mainline/lib/sup/maildir.rb:105:in `poll'
/Users/jdugan/projects/sup/mainline/lib/sup/maildir.rb:90:in `each'
/Users/jdugan/projects/sup/mainline/lib/sup/maildir.rb:90:in `poll'
/Users/jdugan/projects/sup/mainline/lib/sup/poll.rb:155:in `poll_from'
/Users/jdugan/projects/sup/mainline/lib/sup/poll.rb:113:in `do_poll'
/Users/jdugan/projects/sup/mainline/lib/sup/poll.rb:103:in `each'
/Users/jdugan/projects/sup/mainline/lib/sup/poll.rb:103:in `do_poll'
/Users/jdugan/projects/sup/mainline/lib/sup/poll.rb:102:in `synchronize'
/Users/jdugan/projects/sup/mainline/lib/sup/poll.rb:102:in `do_poll'
/Users/jdugan/projects/sup/mainline/lib/sup/util.rb:618:in `send'
/Users/jdugan/projects/sup/mainline/lib/sup/util.rb:618:in `method_missing'
/Users/jdugan/projects/sup/mainline/lib/sup/modes/poll-mode.rb:15:in `poll'
/Users/jdugan/projects/sup/mainline/lib/sup/poll.rb:49:in `poll_with_sources'
/Users/jdugan/projects/sup/mainline/lib/sup/poll.rb:68:in `poll'
/Users/jdugan/projects/sup/mainline/lib/sup/util.rb:618:in `send'
/Users/jdugan/projects/sup/mainline/lib/sup/util.rb:618:in `method_missing'
/Users/jdugan/projects/sup/mainline/bin/sup:223
/Users/jdugan/projects/sup/mainline/lib/sup.rb:78:in `reporting_thread'
/Users/jdugan/projects/sup/mainline/lib/sup.rb:76:in `initialize'
/Users/jdugan/projects/sup/mainline/lib/sup.rb:76:in `new'
/Users/jdugan/projects/sup/mainline/lib/sup.rb:76:in `reporting_thread'
/Users/jdugan/projects/sup/mainline/bin/sup:223
/Users/jdugan/projects/sup/mainline/lib/sup/modes/thread-index-mode.rb:684:in `call'
/Users/jdugan/projects/sup/mainline/lib/sup/modes/thread-index-mode.rb:684:in `__unprotected_load_threads'
/Users/jdugan/projects/sup/mainline/lib/sup/modes/thread-index-mode.rb:625:in `call'
/Users/jdugan/projects/sup/mainline/lib/sup/modes/thread-index-mode.rb:625:in `load_n_threads_background'
/Users/jdugan/projects/sup/mainline/lib/sup.rb:78:in `reporting_thread'
/Users/jdugan/projects/sup/mainline/lib/sup.rb:76:in `initialize'
/Users/jdugan/projects/sup/mainline/lib/sup.rb:76:in `new'
/Users/jdugan/projects/sup/mainline/lib/sup.rb:76:in `reporting_thread'
/Users/jdugan/projects/sup/mainline/lib/sup/modes/thread-index-mode.rb:623:in `load_n_threads_background'
/Users/jdugan/projects/sup/mainline/lib/sup/modes/thread-index-mode.rb:694:in `__unprotected_load_threads'
(eval):12:in `load_threads'
/Users/jdugan/projects/sup/mainline/bin/sup:223
_______________________________________________
sup-talk mailing list
sup-talk@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-talk