[HCoop-Discuss] Reorganizing, people-wise and tech-wise

[Matthias-Christian Ott]

On Thu, Jun 25, 2009 at 04:47:26PM -0400, Adam Chlipala wrote:
> Matthias-Christian Ott wrote:
> > Instead of giving up the idea of a distributed filesystems, we should
> > reevaluate other possible solutions. I have experience with NFS and 9P.
> > NFS kind of worked seems be wide-spread. 9P is primarily present on Plan
> > 9.
> >   
> 
> Can you explain why you think it makes sense to put home directories in 
> a networked filesystem by default? I'm pretty sure most HCoop members 
> don't see any particular benefit from it.

See below.
 
> > I vote against separate machines, my experience at TIP9UG showed that
> > it's quite valuable to have a shared filesystem.
> >   
> 
> Can you elaborate on this?

They have a single fileserver which store all the data and you can
simple log into any of the diskless servers and find the same
environment. These servers just provide CPU power and the filesystem
server provides the storage.

You could load balance, do cloud computing (like Google App Engine or so),
and you don't have a single point of failure, so if one server goes down
(except the fileserver), the members can simply use one of the others.
Moreover, new services don't have to be distributed to the server (like
http, smtp, imap or pop), because they operate on the same set of data,
this saves a lot of administration work and is more flexible. You could
simply PnP new servers.

I have to admit that the benefits are greater if you have a hive mind
cluster with a truely distributed filesystem.
 
> >  
> >   
> >> Here are the steps I can think of for getting from here to there.
> >>
> >> 1. Form a committee of at least 3 members who are responsible for all 
> >> hardware purchases (and ideally most of whom aren't volunteering for 
> >> anything else).  They should determine what we should buy and who we 
> >> should buy it from.
> >> 2. This committee should find either 2 or 3 fairly beefy 1U servers for 
> >> us to buy and colocate at a new provider with rates more in line with 
> >> the average.
> >>     
> >
> > Peer1 wants 75 USD/1U, is that correct?
> >   
> 
> I'm not sure about any prices beyond the $750/mo. we pay now for a 
> quarter cabinet.

Well, it sounds too much anyways.
 
> > Maybe we should choose the location for the servers near someone
> > reliable (who is HCoop member for a long time or so), so that if some
> > hardware stuff has to be fixed, he can go to the hoster.
> >   
> 
> I'd much rather find a colo provider whose techs will fix our stuff for 
> reasonable rates. People move to different cities all the time; we 
> shouldn't depend on having someone in a particular location, unless we 
> can pay this person.

That sounds reasonable, but we should choose a reliable provider with
good support in that case.
 
> >> 5. These admins divide up the work to set up the servers as outlined 
> >> above, documenting everything in our wiki.  We have the main server, the 
> >> member server, and (optionally) the spare server.  I expect that we can 
> >> buy a beefy enough member server that we can handle the current load 
> >> just fine (there are used machines available for under $1000 that have 
> >> more capacity than all 5 of our currently-on machines put together), 
> >> though we would want to start planning immediately for adding new member 
> >> servers when needed.
> >>     
> >
> > But we will keep the current server, am I right? Or are we going to sell
> > at least some of them?
> 
> I think all of our current servers are too underpowered to make sense to 
> colocate, given that recurring costs are based on space. A beefy 1U 
> server takes up as much space as any of our current servers, but we can 
> get a lot more out of it. Also, we have no warranties or service plans 
> for any of our current machines, and we may be able to procure new 
> machines with such plans for reasonable prices. I would opt for selling 
> all of our current servers, once the new stuff has been up and running 
> for a while.

You are perhaps right, if we need less servers the price for hosting
will be less and buying new servers will pay off.

Regards,
Matthias-Christian