[HCoop-Discuss] Planning

[Adam Chlipala]

I've just passed a deadline for a work thing that was keeping me busy.  
I'll continue to be pretty busy through the end of the summer, but I'm 
now expecting to be able to spend a significant amount of time 
organizing our transition to wherever we want to go next with HCoop.

We should have an official HCoop Board of Directors IRC meeting soon, to 
have official votes on our new directions.  I think we need a fair 
amount of additional discussion here first, though.

I'm still favoring switching to a set-up where each non-admin member has 
an account on a single high-capacity machine, with minimal use of 
distributed systems.  My vision is that these machines would be set up 
like beefier versions of what most of us run on home Linux machines.  In 
our previous discussion, we ended up with this wiki page created, where 
we have three people indicating interest in volunteer admin work on such 
a set-up:
    http://wiki.hcoop.net/AdminArea/ListOfVolunteers
Davor, I'm assuming that you're interested in maintaining some 
involvement with admin stuff.  Am I right?  How about you, Clinton?

Like I suggested before, I'd also like to see volunteers for two other 
groups.  We need some people to take charge of figuring out the right 
colocation provider to switch to.  This would be a one-time thing, 
without any expectation of recurring responsibilities.  We also need a 
permanent committee charged with deciding which hardware we should buy 
and who we should buy it from.  Could anyone interested in either of 
these jobs please add himself to the wiki page linked above?

The big high-level issues that remain are
(A) whether we want any kind of networked filesystem or other 
distributed stuff; and
(B) whether we want to take any advantage of virtualization

Both of these have intuitive geek appeal, along with obvious practical 
benefits with infinite resources devoted to admin work.  The way I see 
things now, though, is that neither is a good idea for HCoop.  Both 
require specialized skills and experience that place serious limits on 
our ability to find volunteer admins.  HCoop really isn't targeted at 
people looking for the highest availability or performance; VPSes are 
cheap enough that members who want those things will look to them 
instead.  This isn't to say that we can't provide VPSes some day, but I 
think that we should focus on getting the basics up first.  It would 
also be neat to offer distributed filesystems and other goodies in a way 
that is very unlikely to have any negative impact on members who don't 
want to use them.  AFS definitely has not had that property for us.


As someone suggested previously, I think it would be good to get an idea 
of how the overall member base feels about some of these issues.  In 
particular, I'm proposing sending to hcoop-announce a message containing 
the following.  Please reply here with any suggested changes.  I'll send 
out the announcement some time this Saturday if no one has any 
suggestions by then.
-------------

We're currently planning the right next steps for HCoop, in terms of 
what infrastructure is worth working to provide to members.  It would be 
helpful to our volunteers if you would vote in this poll:
    [URL]

The question is whether or not you are interested in using a networked 
filesystem or other distributed system functionality with your HCoop 
account.  AFS and Kerberos are examples of that kind of functionality 
that we're using now; NFS is a more common example.  An alternative is 
to give each member a traditional UNIX account on a standalone machine.  
We've debated the question some on the hcoop-discuss mailing list, and 
there hasn't been clear agreement.  Here are some pros and cons for the 
different choices.

* Why we might want to keep using distributed systems *

It's convenient to be able to access a common filesystem across HCoop 
machines and from personal machines, as is possible now with AFS.  
Having this infrastructure in place makes it easier for members to 
deploy high-availability distributed applications across multiple HCoop 
systems.

* Why we might want to switch to standalone servers *

Distributed systems are less commonly used and more complex.  It seems 
likely that most of our performance and availability problems from the 
last few years would not have come up or would have been less severe if 
we weren't using AFS.  We have a hard time finding qualified volunteer 
admins for these less common services; we still haven't had anyone with 
significant AFS experience volunteer to make any specific time 
commitment to handling it at HCoop.  Many common UNIX procedures, like 
SSH public key log-in, permissions for web applications, and standard 
usage of cron tabs, require extra work with AFS; with other networked 
filesystems, the situation is similar, with either the same problems or 
with potential security issues if a single machine with access to the 
filesystem is compromised.