Bugs

Since Monkeysphere is using bash, it would be nice to use the shell build in getopts function, instead of the external getopt program. This would reduce an external dependency, which would definitely be better for portability.

---

So it looks like the sh built-in getopts does not include long options (eg. "--expire"). Is it worth getting rid of the long options for this?

---

Why not just get rid of getopts altogether and perform a simple argument-processing loop with bash string tests? We're only invoking getopt in three places, and each invocation is no more complex than three arguments -- and most arguments take a separate parameter, which means that handling tricky arg blobs like -aCxr are not gonna be supported anyway.

It would be nice to make all of the Monkeysphere scripts POSIX compliant, for portability and light-weightedness. Better POSIX compliance would probably at least be better for compatibility with o{ther,lder} versions of bash. Unfortunately there are quite a few bashism at the moment, so this may not be trivial. For instance:

  servo:~/cmrg/monkeysphere/git 0$ checkbashisms -f src/monkeysphere-server 2>&1 | wc -l
  50
  servo:~/cmrg/monkeysphere/git 0$

It looks like the biggest complication for this would be the occasional use of bash arrays.

I would like to know, at INFO (default) log level, when the monkeyspehere makes a "real" modification to my known_hosts file; that is, when it adds or deletes a key.

Apparently this is hard because monkeysphere is currently configured to delete all keys and then add good keys, so a key added for the first time seems to the monkeysphere very similar to a key re-added ten seconds after last login.

Still, from a UI perspective, I want to know what monkeysphere is doing.

---

It looks like jrollins committed a change for reporting at INFO level when a host key gets added by the monkeysphere: 2459fa3ea277d7b9289945748619eab1e3441e5c

When i connect to a host whose key is not already present in my known_hosts file, i get the following to stderr:

ms: * new key for squeak.fifthhorseman.net added to known_hosts file.

This doesn't fully close this bug, because we aren't notifying on key deletion, afaict.

---

So current log level DEBUG will output a message if the known host file has been modified. If the issue is that you want to know at the default log level everytime the known_hots file is modified, then we should just move this message to INFO instead of debug, and then maybe remove the message that I added above. I was under the impression that the issue was more about notification that a new key was added to the known_hosts file, and therefore the new INFO message above fixed that problem. Should we do this instead?

In general, more verbose log levels do tell the user what the monkeysphere is doing. Moving to DEBUG log level will tell you pretty much everything that happens. I do not think that this should be the default log level, though.

---

I wouldn't want to see an extremely verbose default log level. But i do think that saying something like "key blah blah blah was stripped from your known_hosts file because it was expired" (for example) would be useful. I think this case would occur infrequently enough that it is worth reporting in the UI at the regular log level.

--dkg

In ~/.ssh/config, i have:

HashKnownHosts No

But when monkeysphere-ssh-proxycommand adds new hosts to ~/.ssh/known_hosts, they appear to be added in a hashed form, instead of in the clear.

fwiw: i'm using OpenSSH 5.1p1 on a debian lenny system (backported from sid)

---

I can confirm this too (I'm running openssh-client 1:4.7p1-12)

-- Jamie (Jam Jam)

---

There is absolutely no attempt by any monkeysphere utility to parse any ssh or sshd config file. This will probably need to be delt with down the line, but it's not a particular easy task at the moment.

-- Big Jimmy.

---

I've posted to the openssh-unix-dev list to see if there is a possibility of openssh making our lives easier here, but i haven't had much of a response yet.

--dkg

---

For some reason this didn't get mentioned in this bug earlier, but there is a monkeysphere config variable about hashing known_hosts lines, which is set to true by default (to be in sync with the Debian openssh-client package).

I think this bug is really more about the fact that monkeysphere does not parse the ssh config files for any directives relavent to what the monkeysphere is doing. I'm changing the name of this bug to reflect what the real issue is.

-- Big Jimmy.

/var/lib/monkeysphere/gnupg-host/ is root-owned, and the public keyring in that directory is controlled by the superuser.

We currently expect the monkeysphere user to read from (but not write to) that keyring. But using a keyring in a directory that you don't control appears to trigger a subtle bug in gpg that has been unresolved for quite a long time.

With some of the new error checking i'm doing in monkeysphere-server, typical operations that involve both keyrings as the non-privileged user can fail with an error message like:

gpg: failed to rebuild keyring cache: file open error

Running the relevant operation a second time as the same user usually lets things go through without a failure, but this seems like it would be hiding a bug, rather than getting it fixed correctly.

Are there other ways we can deal with this problem?

--dkg

Here is an example when using monkeysphere-server add-identity-certifier on a host with a newly-installed monkeysphere installaton. Note that running the same command a second time works as expected:

 0 pip:~# monkeysphere-server c+ 0EE5BE979282D80B9F7540F1CCD2ED94D21739E9
 gpg: requesting key D21739E9 from hkp server pool.sks-keyservers.net
 gpg: key D21739E9: public key "Daniel Kahn Gillmor <dkg@fifthhorseman.net>" imported
 gpg: can't create `/var/lib/monkeysphere/gnupg-host/pubring.gpg.tmp': Permission denied
 gpg: failed to rebuild keyring cache: file open error
 gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
 gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
 gpg: next trustdb check due at 2009-03-30
 gpg: Total number processed: 1
 gpg:               imported: 1  (RSA: 1)
 Could not receive a key with this ID from the 'pool.sks-keyservers.net' keyserver.
 255 pip:~# monkeysphere-server c+ 0EE5BE979282D80B9F7540F1CCD2ED94D21739E9
 gpg: requesting key D21739E9 from hkp server pool.sks-keyservers.net
 gpg: key D21739E9: "Daniel Kahn Gillmor <dkg@fifthhorseman.net>" not changed
 gpg: Total number processed: 1
 gpg:              unchanged: 1

 key found:
 pub   4096R/D21739E9 2007-06-02 [expires: 2012-05-31]
       Key fingerprint = 0EE5 BE97 9282 D80B 9F75  40F1 CCD2 ED94 D217 39E9
 uid       [ unknown] Daniel Kahn Gillmor <dkg@fifthhorseman.net>
 uid       [ unknown] Daniel Kahn Gillmor <dkg@openflows.com>
 uid       [ unknown] Daniel Kahn Gillmor <dkg@astro.columbia.edu>
 uid       [ unknown] Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>
 uid       [ unknown] [jpeg image of size 3515]
 sub   2048R/4BFA08E4 2008-06-19 [expires: 2009-06-19]
 sub   4096R/21484CFF 2007-06-02 [expires: 2012-05-31]

 Are you sure you want to add the above key as a
 certifier of users on this system? (y/N) y
 gpg: key D21739E9: public key "Daniel Kahn Gillmor <dkg@fifthhorseman.net>" imported
 gpg: Total number processed: 1
 gpg:               imported: 1  (RSA: 1)
 gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
 gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
 gpg: next trustdb check due at 2009-03-30
 gpg (GnuPG) 1.4.9; Copyright (C) 2008 Free Software Foundation, Inc.
 This is free software: you are free to change and redistribute it.
 There is NO WARRANTY, to the extent permitted by law.


 pub  4096R/D21739E9  created: 2007-06-02  expires: 2012-05-31  usage: SC  
                      trust: unknown       validity: unknown
 [ unknown] (1). Daniel Kahn Gillmor <dkg@fifthhorseman.net>
 [ unknown] (2)  Daniel Kahn Gillmor <dkg@openflows.com>
 [ unknown] (3)  Daniel Kahn Gillmor <dkg@astro.columbia.edu>
 [ unknown] (4)  Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>
 [ unknown] (5)  [jpeg image of size 3515]


 pub  4096R/D21739E9  created: 2007-06-02  expires: 2012-05-31  usage: SC  
                      trust: unknown       validity: unknown
  Primary key fingerprint: 0EE5 BE97 9282 D80B 9F75  40F1 CCD2 ED94 D217 39E9

      Daniel Kahn Gillmor <dkg@fifthhorseman.net>
      Daniel Kahn Gillmor <dkg@openflows.com>
      Daniel Kahn Gillmor <dkg@astro.columbia.edu>
      Daniel Kahn Gillmor <dkg-debian.org@fifthhorseman.net>
      [jpeg image of size 3515]

 This key is due to expire on 2012-05-31.
 Please decide how far you trust this user to correctly verify other users' keys
 (by looking at passports, checking fingerprints from different sources, etc.)

   1 = I trust marginally
   2 = I trust fully


 Please enter the depth of this trust signature.
 A depth greater than 1 allows the key you are signing to make
 trust signatures on your behalf.


 Please enter a domain to restrict this signature, or enter for none.


 Are you sure that you want to sign this key with your
 key "ssh://pip.fifthhorseman.net" (9B83C17D)

 The signature will be marked as non-exportable.


 gpg: can't create `/var/lib/monkeysphere/gnupg-host/pubring.gpg.tmp': Permission denied
 gpg: failed to rebuild keyring cache: file open error
 gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
 gpg: depth: 0  valid:   1  signed:   1  trust: 0-, 0q, 0n, 0m, 0f, 1u
 gpg: depth: 1  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 1f, 0u
 gpg: next trustdb check due at 2009-03-30

 Identity certifier added.
 0 pip:~#

The current monkeysphere Makefile has a "tarball" target, which produces the "upstream tarball". Unfortunately, it is not idempotent. That is, if you run it twice in a row (without changing any other source), the second .orig.tar.gz file is bytewise different from the first.

We should fix this so that the tarball generated is the same at least as long as no local file has been touched.

--dkg

What if everything that's done in the package post-installation scripts (aside from maybe the creation of the monkeysphere user itself) was done with a single call to something like

monkeysphere-server setup

This would make things more obvious to folks installing from source directly, and put less maintenance load on porters. The end of monkeysphere-server setup could also invoke monkeysphere-server diagnostics to get the admin pointed in the right direction.

Think of this as a sort of automated "Getting Started" documentation.

Of course, a hypothetical full setup command would do things like gen-key, auto-modify sshd_config, etc. We wouldn't want to do those things automatically without the guiding hand of the local sysadmin.

But perhaps we could even smooth that process with:

monkeysphere-server setup --full

I'd like to know what other folks think about these possibilities. Would either of these be useful? Are they confusing? Could they be clarified?

--dkg

---

I'm not sure how I feel about this idea. I feel like it should just be the job of the package to setup the initial server environment. I don't really feel like the admin should have to worry about it. But then again, I can sort of see it from the point of view of someone just installing from source (but who the hell really does that anymore anyway?).

I'm also sort of mixed about the setup --full idea as well. At first I thought that it wasn't a good idea, and that I didn't like the idea of monkeysphere monkeying around with the config files of other packages (ie. ssh). However, once I started to think about setting up monkeysphere on lots of servers, I started to think that it's maybe a good idea. It might be good to have a single command that would just end with the server being on the monkeysphere:

• generate the server key
• modify sshd to point to it
• restart ssh
• publish the key to the keyserver

So I'm starting to think that this might be a good idea. Also curious what other think.

-- jrollins

OpenSSH allows users to control the capabilities granted to remote key-based logins by supplying options that should limit the use of the key.

For example, specifying no-pty means that sshd should not allocate a pseudo-terminal for sessions created based on an authentication with that key.

It is unclear if it is possible to do this sort of limiting in ~/.monkeysphere/authorized_user_ids, and if it is possible, how you'd actually do it.

--dkg

If you have a revoked authentication subkey in your keyring, monkeysphere gen-subkey thinks that I have an authentication subkey already, which I do, but it probably shouldn't care about it, since it is revoked:

     21:30@pond> monkeysphere gen-subkey F67E2A5D1CF2D62A
 An authentication subkey already exists for key 'F67E2A5D1CF2D62A'.
 Are you sure you would like to generate another one? (y/N)

However: this key was revoked on 2008-04-28 by DSA key 1CF2D62A Micah Anderson micah@riseup.net sub 1024R/866F47D3 created: 2008-02-25 revoked: 2008-04-28 usage: A

I can continue to create a new authorization subkey, so its not a blocker or anything (I suppose I could also delete the revoked key from my keyring as well, although thats less than ideal).

It seems like the secret keyring doesn't mention that it has been revoked, so probably monkeysphere needs to be looking at gpg's computed validity from the public keyring instead of the secret keyring to be able to get the "r" flag from field 2, in addition to the "e" flag from field 12.

---

So the problem is that there is no field 2 for secret keys. From /usr/share/doc/gnupg/DETAILS.gz:

  2. Field:  A letter describing the calculated trust. This is a single
         letter, but be prepared that additional information may follow
     in some future versions. (not used for secret keys)

Why would secret keys not have this field? They have validity too, right? This doesn't make any sense. I verify that indeed there is no output in field 2 for secret keys. I would say this is a bug in gpg, but it's clearly done on purpose. Any ideas?

-- jrollins

In monkeysphere-server, the gpg_authentication function, and consequently the gpg_authentication_cmd, currently requires that all arguments be put in a single quoted argument, eg:

      gpg_authentication "--list-key --with-colons --with-fingerprint 0x${keyID}!"

This is obviously a little lame, but it seems to be necessary to do the necessary argument passing from the function, to the su function called as the monkeysphere user that controls the gpg authentication keyring.

I'm not sure how to fix it. I think the problem is mostly in how arguments are passed to su.

-- Big Jimmy.

We should publish the various monkeysphere man pages in browsable form somewhere under http://web.monkeysphere.info/. Ideally, this would be updated automatically from the sources for the official man pages themselves.

This strikes me as an ikiwiki subproject (implementing a man2html wiki compilation language perhaps?).

Interestingly, ikiwiki's own man page appears to be written in markdown and then converted to nroff.

It appears that the monkeysphere-server revoke-hostname function will occasionaly revoke the wrong hostname. I say occasionally, but it seems to be doing it pretty consistently for me at the moment:

servo:~ 0$ sudo monkeysphere-server n- servo.finestructure.net
The following host key user ID will be revoked:
  ssh://servo.finestructure.net
Are you sure you would like to revoke this user ID? (y/N) y
gpg (GnuPG) 1.4.9; Copyright (C) 2008 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Secret key is available.

pub  1024R/9EEAC276  created: 2008-07-10  expires: never       usage: CA  
                     trust: ultimate      validity: ultimate
[ultimate] (1)  ssh://localhost.localdomain
[ultimate] (2). ssh://servo.finestructure.net
[ revoked] (3)  ssh://jamie.rollins
[ revoked] (4)  asdfsdflkjsdf
[ revoked] (5)  ssh://asdfsdlf.safsdf
[ revoked] (6)  ssh://bar.baz
[ revoked] (7)  ssh://foo.bar
[ revoked] (8)  ssh://


pub  1024R/9EEAC276  created: 2008-07-10  expires: never       usage: CA  
                     trust: ultimate      validity: ultimate
[ultimate] (1)* ssh://localhost.localdomain
[ultimate] (2). ssh://servo.finestructure.net
[ revoked] (3)  ssh://jamie.rollins
[ revoked] (4)  asdfsdflkjsdf
[ revoked] (5)  ssh://asdfsdlf.safsdf
[ revoked] (6)  ssh://bar.baz
[ revoked] (7)  ssh://foo.bar
[ revoked] (8)  ssh://

Please select the reason for the revocation:
  0 = No reason specified
  4 = User ID is no longer valid
  Q = Cancel
(Probably you want to select 4 here)
Enter an optional description; end it with an empty line:
Reason for revocation: User ID is no longer valid
Hostname removed by monkeysphere-server 2008-08-16T17:34:02

pub  1024R/9EEAC276  created: 2008-07-10  expires: never       usage: CA  
                     trust: ultimate      validity: ultimate
[ revoked] (1)  ssh://localhost.localdomain
[ultimate] (2). ssh://servo.finestructure.net
[ revoked] (3)  ssh://jamie.rollins
[ revoked] (4)  asdfsdflkjsdf
[ revoked] (5)  ssh://asdfsdlf.safsdf
[ revoked] (6)  ssh://bar.baz
[ revoked] (7)  ssh://foo.bar
[ revoked] (8)  ssh://

gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0  valid:   1  signed:   2  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: depth: 1  valid:   2  signed:   0  trust: 0-, 0q, 0n, 0m, 2f, 0u
gpg: next trustdb check due at 2012-01-07
sec   1024R/9EEAC276 2008-07-10
      Key fingerprint = C094 43E0 6882 8BE2 E9AD  516C 45CF 974D 9EEA C276
uid                  ssh://servo.finestructure.net
uid       [ revoked] ssh://localhost.localdomain
uid       [ revoked] ssh://jamie.rollins
uid       [ revoked] asdfsdflkjsdf
uid       [ revoked] ssh://asdfsdlf.safsdf
uid       [ revoked] ssh://bar.baz
uid       [ revoked] ssh://foo.bar
uid       [ revoked] ssh://

NOTE: User ID revoked, but revokation not published.
Run 'monkeysphere-server publish-key' to publish the revocation.
servo:~ 0$

Clearly this is unacceptable. gpg does not let you can't specify a uid to revoke from the command line. The uid revokation can only be done through edit-key. We do edit-key scripting in other contexts, but to revoke a user id you have to specify the uid by "number". We currently try to guess the number from the ordering of the output of list-key. However, this output does not appear to coincide with the ordering in edit-key. I don't have a good solution or fix at the moment. Suggestions are most welcome. It may just require some trial and error with edit-key to come up with something workable.

This underlines the problem that gpg is currently not very well suited for manipulating gpg keyrings non-interactively. It's possible that I just haven't figured out how to do it yet, but it's not very clear if it is possible. It would be nice to have some alternate tools to use.

-- Big Jimmy.

At the moment, the only tool we have to export passphrase-locked secret keys from the GPG keyring is gpg itself (and gpg2, which has roughly the same behavior).

As a result, we have the seckey2sshagent hack, which is unfriendly and awkward to use.

Ideally, openpgp2ssh would be able to convert passphrase-locked secret keys into clean subkeys. However, i've tried to do this via GnuTLS, and that library is not ready for this.

OpenCDK, which is the component of GnuTLS which reads OpenPGP-style keys, cannot cope with encrypted secret key material. I have had some success in getting GnuTLS's OpenCDK to accept the existence of encrypted secret key packets, i learned that OpenCDK as included in GnuTLS is incapable of dealing with the encrypted packets themselves.

Some possible resolutions:

---

If we can assume that the passphrase-encrypted key we want to use is actually a subkey, and if we could fix GnuTLS to ignore the use of the "gnu-dummy S2K" produced by gpg --export-secret-subkeys for the primary key, then something like the following script should actually work for reasonable values of $KEYID:

TMPDIR=$(mktemp -d)
umask 077
mkfifo "$TMPDIR/passphrase"
kname="MonkeySphere Key $KEYID"
mkfifo "$TMPDIR/$kname"
ssh-askpass "Please enter the passphrase for MonkeySphere key $KEYID" >"$TMPDIR/passphrase" &
gpg  --passphrase-fd 3 3<"$TMPDIR/passphrase" \
  --export-options export-reset-subkey-passwd,export-minimal,no-export-attributes \
  --export-secret-subkeys "$KEYID"\! | openpgp2ssh "$KEYID" > "$TMPDIR/$kname" &
(cd "$TMPDIR" && ssh-add -c "$kname")
rm -rf "$TMPDIR"

Good news! I've crafted a patch for GnuTLS to enable it to read exported subkeys using this GNU extension, so if we can get it incorporated into upstream (and/or into debian), we have a possible solution, as long as the authentication key is a subkey, and not a primary key.

As of version 0.11-1, monkeysphere subkey-to-ssh-agent implements this particular strategy (and fails cleanly if the version of GnuTLS present doesn't support the GNU dummy S2K extension).

---

Ben Laurie and Rachel Willmer's OpenPGPSDK is a candidate: this is a C-based library that intends to implement RFC 4880 functionality.

We could potentially re-write openpgp2ssh using this library, and it should be able to handle everything we need from the OpenPGP side (though it might need to be re-linked to OpenSSL to handle PEM-encoded exports.

Concerns:

• OpenPGPSDK is not in debian yet, and doesn't currently (2008-08-13) build with gcc 4.2 or 4.3.

• OpenPGPSDK uses the apache license and appears to link to OpenSSL, which has a GPL-incompatible license. I think this would mean that openpgp2ssh could not remain GPL (though the rest of the monkeysphere could).

---

We could try to use perl. The last time i checked, the pure-perl OpenPGP implementations all depended on Math::PARI, which is not in debian. The most likely candidate is Crypt::OpenPGP, despite some bugginess.

Concerns:

• the aforementioned buggy reviews

• there's a lot of dependency chasing to get anything like this available in debian.

---

Other alternatives?

---

Can this bug be closed? dkg reported in a comment for a related bug:

Version 0.11-1 now has the monkeysphere subkey-to-ssh-agent
subcommand, which works cleanly in the presence of a
functionally-patched GnuTLS.

---

Even with the patched GnuTLS, monkeysphere currently can't currently deal with passphrase-locked primary keys. I've changed the title of this bug, but i'd like to keep it open until we are able to deal with that. The other comments here seem still quite relevant to that need.

I've changed the title of this bug to reflect the narrowed scope.

   --dkg

When i try to generate a key on a headless machine (no kbd, no mouse, no Human Input Device (HID) at all), monkeysphere gen-key hangs for a very long time (over an hour so far!) during the generation process, particularly at this point:

ms: generating server key...

Not enough random bytes available.  Please do some other work to give
the OS a chance to collect more entropy! (Need 197 more bytes)

And sure enough, there really is very little entropy in these systems at the time requested:

0 chomsky:~# cat /proc/sys/kernel/random/entropy_avail 
32
0 chomsky:~#

It's not clear to me how to increase the entropy available to the kernel without an HID.

I've seen this happen on two machines now in the last week, and was able to resolve it on the first one by plugging in a keyboard and "massaging" it. This won't work for a machine that's out of physical range, and has no keyboard to be plugged in anyway.

One thing that might help is to suggest that the system administrator install a package like bsdgames and play console-based games as a non-privileged user, since that seems to feed the entropy count somewhat.

When executing monkeysphere-server add-identity-certifier across a link without a pseudo-terminal, it behaves oddly (prompts are created that are only halfway-readable, gpg gives error messages about lacking access to a /dev/tty, etc.

You can try this directly if you have remote ssh access to the superuser on a monkeysphere-enabled host, assuming that $GPGID is set to the full fingerprint of a key you want to add as a trusted identity certifier:

ssh root@example.org monkeysphere-server add-identity-certifier $GPGID

Compare this behavior with:

ssh -t root@example.org monkeysphere-server add-identity-certifier $GPGID

Consider the following snippet in ~/.ssh/config:

 Host foo
 HostKeyAlias bar

for a host which is not participating in the monkeysphere.

For such a host, when using monkeysphere-ssh-proxy-command, the public keyservers will be queried on each attempted ssh connection (even after a successful connection).

This appears to be because:

• ssh itself will write a line to ~/.ssh/known_hosts, but it will be labeled with bar because of the HostKeyAlias.

• monkeysphere won't be able to find any mention of it in the keyring (it's not in the monkeysphere)

• monkeysphere-ssh-proxycommand won't be able to find it in the known_hosts file because it looks for foo, which is never matched.

excessive keyserver querying is bad behavior, because it causes delays for the users, and puts excessive load on the public keyserver infrastructure.

How can we resolve this?