[sup-devel] [issue117] Problem with "special" characters (space etc.) in source uri (filesystem path).

[anonymous <sup-bugs@masanjin.net>]

New submission from anonymous:

Hey there,

sup has problems with "non-standard", but allowed, characters in the source uri.
My example: I have maildir storages I want to add, and they contain spaces
(which are allowed characters on the filesystem). Here's the error message:

===>
$ sup-add 'maildir:~/maildir-Mail/Inbox/Dresden-Initiativen (Tauschnetz,
Evoluzzer, Vernetzungstreffen ...)'
[Thu Aug 19 09:47:57 +0200 2010] Flushing Xapian updates to disk. This may take
a while...
/usr/lib/ruby/1.8/uri/common.rb:436:in `split': bad URI(is not URI?):
maildir:~/maildir-Mail/Inbox/Dresden-Initiativen (Tauschnetz, Evoluzzer,
Vernetzungstreffen ...) (URI::InvalidURIError)
        from /usr/lib/ruby/1.8/uri/common.rb:485:in `parse'
        from /usr/lib/ruby/1.8/uri/common.rb:608:in `URI'
        from /usr/bin/sup-add:94
        from /usr/bin/sup-add:86:in `each'
        from /usr/bin/sup-add:86
<===

I can circumvent that problem by adding the line marked in the following code
with a comment into /usr/bin/sup-add (it was inserted after line 86 in my
version of the code):

===>
begin
  Redwood::SourceManager.load_sources

  ARGV.each do |uri|
    uri=URI.encode(uri) ### <--- Added this line in order to encode "special"
characters within the uri.
    labels = $opts[:labels] ? $opts[:labels].split(/\s*,\s*/).uniq : []

    if !$opts[:force_new] && Redwood::SourceManager.source_for(uri)
      say "Already know about #{uri}; skipping."
      next
    end
<===
(I don't know anything about ruby; I got this idea from a similar problem with
another program by searching the web.)

This makes sup-add work, it encodes the non-standard characters (like "%20" for
" "). They appear as such in sources.yaml. But this breaks another part:
sup-sync now looks for files containing "%20" instead of " " on the disk, which
obviously don't exist:

===>
$ sup-add 'maildir:~/maildir-Mail/Inbox/Dresden-Initiativen (Tauschnetz,
Evoluzzer, Vernetzungstreffen ...)'
Adding
maildir:~/maildir-Mail/Inbox/Dresden-Initiativen%20(Tauschnetz,%20Evoluzzer,%20Vernetzungstreffen%20...)...
[Thu Aug 19 09:45:29 +0200 2010] Flushing Xapian updates to disk. This may take
a while...

# Works.

$ cat ~/.sup/sources.yaml
---
- !masanjin.net,2006-10-01/Redwood/Maildir
  uri:
maildir:~/maildir-Mail/Inbox/Dresden-Initiativen%20(Tauschnetz,%20Evoluzzer,%20Vernetzungstreffen%20...)
  cur_offset:
  usual: true
  archived: false
  id: 1
  labels: []

  mtimes:
    cur: 1970-01-01 01:00:00 +01:00
    new: 1970-01-01 01:00:00 +01:00

# "Special" characters are Encoded.

$ sup-sync
Scanning
maildir:~/maildir-Mail/Inbox/Dresden-Initiativen%20(Tauschnetz,%20Evoluzzer,%20Vernetzungstreffen%20...)...
[Thu Aug 19 09:45:59 +0200 2010] WARNING: problem getting messages from
maildir:~/maildir-Mail/Inbox/Dresden-Initiativen%20(Tauschnetz,%20Evoluzzer,%20Vernetzungstreffen%20...):
/home/felics/maildir-Mail/Inbox/Dresden-Initiativen%20(Tauschnetz,%20Evoluzzer,%20Vernetzungstreffen%20...)/cur
not a directory
Scanned 0, added 0, updated 0 messages from
maildir:~/maildir-Mail/Inbox/Dresden-Initiativen%20(Tauschnetz,%20Evoluzzer,%20Vernetzungstreffen%20...).
[Thu Aug 19 09:45:59 +0200 2010] Flushing Xapian updates to disk. This may take
a while...

# That directory does not exist. How should it, it has " " within it's name, not
"%20".
<===

Next try: Exchanging back in sources.yaml all "%20" to " ". But that breaks
again sup-sync (the "URI::InvalidURIError" as above). This I was unable to fix,
since I did not find a point where a simple insertion of a URI.encode statement
could help.

This problems are also there for paths containing only "normal" characters and
spaces, i.e. no parentheses. And there are also other special characters that
make problems: '<', '>', '"'. Probably umlauts.

Clean solution would be to make sup safe for any character (in any encoding)
that is allowed to appear within the sources.

For me, that's an KO-argument not to use sup, since I still need that directory
structure to represent some email organisation on filesystem level, and I will
not rename every single directory just for sup (it's part of a bigger email setup).

Bye,
Felix.

----------
keyword: encoding, maildir
messages: 271
nosy: anonymous
priority: bug
ruby_version: 1.8.7 (2008-08-11 patchlevel 72) [i486-linux] (Debian)
status: unread
sup_version: 0.11-2 (Debian)
title: Problem with "special" characters (space etc.) in source uri (filesystem path).

_________________________________________
Sup issue tracker <sup-bugs@masanjin.net>
<http://masanjin.net/sup-bugs/issue117>
_________________________________________
_______________________________________________
Sup-devel mailing list
Sup-devel@rubyforge.org
http://rubyforge.org/mailman/listinfo/sup-devel