Blue Mass Group

Reality-based commentary on politics.

Save Soapblox — a fundraiser

By

As you all know, Soapblox was seriously hacked a couple of days ago.  Much of the Soapblox network (including BMG) was off-line for several hours, and for a while there it looked as though it — and we — might never come back.

Fortunately, the immediate crisis has been averted.  But the episode revealed major vulnerabilities in Soapblox, and made obvious what probably should have been obvious before: one guy managing Soapblox in his spare time was a disaster waiting to happen.  Paul Preston, who developed and runs Soapblox, has done extraordinary things with it — it’s a powerful and inexpensive platform that now runs dozens of blogs, including most state-level community blogs as well as national operations like Open Left and Pam’s House Blend.  But in a sense, Paul is a victim of his own success.  The Soapblox project is now simply too big, and too important, to be managed in the haphazard fashion that has prevailed until now.

Goal Thermometer Inevitable conclusion: we need to raise money, both to handle the short-term fixes that the hack showed to be necessary, and to set a longer-term foundation for a stable and secure blogging infrastructure.  

And so we are asking all of you to consider a small donation to preserve and protect Soapblox.  This is national effort — there is a front-page post at Daily Kos and at Open Left by Chris Bowers (reprinted below) making the pitch, as well as similar asks on state-level blogs around the country.  Chris’s post explains in detail the importance of Soapblox (which you already know), and breaks down the plan going forward and the financial requirements.  Here’s the short version from Chris’s post:

Here is what Soapblox needs in the immediate short-term to become safe and secure once again:

–Recharge ten servers

–Perform a full security audit of the SoapBlox server/unix infrastructure to prevent hackers from gaining access

–Ensure all backup processes are working and functional to guarantee that if hacking happens, data is preserved

–Perform a security audit on the SoapBlox code itself so that hackers cannot exploit the SoapBlox code itself.

–Migrate to new, secure servers

The good news is that, in addition to restoring full service for Soapblox, Paul has already found a system administrator who lives in his area and is able to help. All of the work listed above is currently underway. Here is what it will cost:

–Recharging ten servers at $100 apiece: $1,000

–Purchasing new, secure severs, and migrating the data: $8,000

–One month of full-time work at $50 / hour in order to complete all of the tasks listed above: $8,400

So, for a total of $17,400, we can secure Soapblox, and ensure that yesterday’s dangerous attack can never be replicated. Let’s make this happen.

Now, I know that many of you harbor worries about Soapblox going forward, even if this fundraiser is successful.  So do we.  But the fact is that, as of right now, Soapblox is an essential part of the progressive netroots infrastructure in this country.  There may be other options out there for us, and for other blogs.  But going to any of those options poses technical and financial challenges that require a lot of study before undertaking.  Soapblox needs help now, and it’s important to us both here in MA and around the country that Soapblox gets the help it needs.

So we hope you’ll participate, either by a one-time contribution, or — even better — by committing to a monthly donation.  Contributing just $5 a month for the next year would be a huge, huge help.

Thank you.

(Technical note: the fundraiser is being held on ActBlue via BlogPac, which is a federally-registered PAC that Chris Bowers, Matt Stoller, and others set up to fund progressive infrastructure projects.  Among other things, BlogPac has underwritten the hosting costs of many state-level blogs (including BMG) for the last couple of years.)

Please share widely!
fb-share-icon
Tweet
0

Comments

  1. The only way I know how to lead is from the front.  So I made my donation.  Given that my “reserves” are so minimal, if I can do this, you can do this.  I don’t ask anyone else to do what I am not willing to do.

  2. what are plans for having more people than Paul running the platform?  Pam Spaulding raised a great point elsewhere: what happens in the unfortunate event that something serious happens to Paul?  If he holds all the keys, all the backups in the world won’t help anyone.

    <

    p>Also, I do hope that the backups will not reside in the same location as the main servers.  I’ve seen this done before, to great tragedy if the building burns or crumbles.  Has BMG obtained backups of its own data yet?

    <

    p>I decided to become a voluntary subscriber to Pam’s House Blend.  Should have done so long ago, but the soapblox thing really snapped me to as to my responsibilities towards something I consider a valuable resource.  Anyway, I figure since she has a lot more insight into this sort of thing than I do, she can allocate my subscription fee (or not) towards this fundraiser of she wants to.

    <

    p>I must say that while I appreciate the idea of the fundraiser to keep things moving if the alternative is worse, doesn’t Paul have fiscal responsibility here?  Even Paul gets a bailout for failing to run his business responsibly.  It just makes my head hurt.  If he can’t do the job alone and responsibly, he needs to increase his fees so that he can hire a 2nd person to help administer the thing.  And not just during the fix, for the next month, but forever.

    • Paul has had a few people helping with sysadminning and coding in little ways. If he was hit by a bus, soapblox wouldn’t just die.

  3. if sufficient/all of the code was going to be released to public domain in one of any number of different licenses.

    <

    p>Frankly, the problem is that so much hinges on Paul.  He did great work to be sure, but clearly he’s not perfect, in code or in management.  The code needs more eyeballs, and I’m not talking about another two eyeballs for $18 large.

    <

    p>

    <

    p>Right now, Paul is the lynch pin.  Robust projects simply don’t have single points of failure, and until Paul brings other folks in, it’s destined to fail again.

    • From Chris’s post:

      <

      p>

      Paul has already found a system administrator who lives in his area and is able to help. All of the work listed above is currently underway.

      <

      p>As for open-sourcing the code, that’s under discussion, and may well happen.  Many others have raised the same issue, and my guess is that it will happen, though I can’t say when.

      • much of the expense is equipment-related — new servers and data migration, e.g.  The rest is to pay the new sys admin to do that work.  Doesn’t seem unreasonable to me — someone has to do it, and why should they do it for free?

        <

        p>”More eyeballs” will of course come with open-sourcing the code.  I hope they do that.  But the immediate need is to secure the project as it now exists.  And that’s worth doing.

      • to mean that he found someone to help with the short-term fixes.  Are you certain that he has found someone to help long-term?

        • The long-term thing remains under discussion, both because we don’t yet know how much $$ can actually be raised, and also because the open-source decision isn’t yet made AFAIK.

          • two more eyeballs — as a direct hire — doesn’t cut the mustard.  There needs to be an organization, so there’s not a single point of failure.

            <

            p>I see no point of investing in the project until it becomes clear that the investment will bring a project that doesn’t rely on him individually to remain stable and useful.

            • Short version: if soapblox fails now, it’s a catastrophe for us, and for the progressive blogosphere.  Avoiding that immediate meltdown is worth raising a bit of money.  We can worry about the other stuff in due course.

    • Public domain works have no restrictions on them.

      <

      p>Open source, Free, etc., software, have licenses which DO have various restrictions.  Most of these are restrictions that are intended to keep the code open, but they are restrictions nonetheless.

      <

      p>While I think that Paul should really consider opening up the code, I do not think he should make it public domain.  A small but important distinction.

      <

      p>Brian

  5. I hate to say it, but I don’t think this is actually a solution to the problem.  For a complex software project like this, you either need really deep pockets to hire several people full time or a large community of developers.  You’d be better off to use the money to migrate to a well-supported CMS like Drupal, Joomla, or Plone, in my opinion.

      1. We at BMG can’t raise all this money ourselves.  This is a national effort.

      <

      p>2. Can you guarantee the preservation of four years of archives — nearly 15,000 posts, over 150,000 comments, nearly 5,000 user accounts, etc. — if we switch to something else?

      <

      p>3. Who is going to administer the site at a new location?  None of us is a techie, so we have to have someone else do it.

      <

      p>4. Soapblox may not be perfect (in fact, it clearly is not), but it does an awful lot pretty well.  Do all of those other systems offer the functionality of Soapblox?  Did you read over the comments in the previous thread, in which several discussions over the relative merits of various of these other options have been discussed?

      <

      p>There are no simple answers here.

      • with a one-man shop.  You’re stuck.  No data migration path.  He’s got you by the balls — so much so that instead of raising money for political races or charities or whatever, you’re raising money for a project that, when complete, will still have you by the balls.

        <

        p>There’s no easy solution, to be sure… but getting him back up and running without reassurances that you won’t be stuck in the same rut later seems awfully short sighted to me.

        • Your point about needing an organization is well taken.  But that begs the question: who is going to pay for it?  

          <

          p>There are short term needs, and longer term needs.  We are trying to address the short term needs with this fundraiser.  The longer term issues will be sorted out soon.  Right now, we have an immediate problem that demands an immediate solution that will cost about $17,000.

          • You want money to fix it, cool.  But if it’s our money, we’ve got conditions on the fix… like make sure you can’t personally let it get screwed up again.

            <

            p>Otherwise, how soon do you have good money chasing bad?

  6. from Paul

    <

    p>

    To all customers, bloggers, and media.

    Starting on Tuesday, some of the SoapBlox servers were infiltrated by a currently unknown source for unknown reasons.

    SSH access was acquired by the hackers, where they were able to install port scanning scripts on to the machines.  SoapBlox servers were then being used to scan other servers across the Internet, looking for vulnerabilities.  

    The disruption of service was caused by our ISP disconnecting the affected servers when they became aware of the port scanning.

    The attack was limited in scope, effecting 4 separate SoapBlox servers.  While around 25 SoapBlox sites were offline, thousands of people were affected.

    Steps are being taken now to identify who did this, and proper legal action will be taken.  If you have any insight on to who perpetrated this attack, please let us know.

    At this time, all services are returned to normal.

    We have many wonderful people now volunteering to ensure this doesn’t happen again. Clean servers are being created, and existing sites will be migrated shortly on to these more secure servers.

    Discussions are currently underway on how to best provide the SoapBlox service, continually improve it, and keep it funded in a way that keeps everything running smoothly.  Soon we will be establishing a way for you to help provide whatever you are willing to keep SoapBlox–and a large chunk of the progressive blogosphere–safe, secure and constantly improving.  

    Please monitor SoapBlox.net for future announcements, and feel free to contact us at soapblox@gmail.com with any ideas or suggestions you might have.  Everything is on the table.

    I apologize with all of my heart for the events of the past two days–from the lack of proper communication, to not seeking help that so many of you are willing to give earlier.  

    We’ve all invested so much time and effort in the software and communities, and I am nothing but humbled by the outpouring of support being given.  It shows the true strength of the progressive blogosphere, and these events will lead to a stronger, better SoapBlox platform for all.

    Paul Preston
    President,
    SoapBlox Network, Inc.

    • The hackers exploited an SSH vulnerability (unlikely) or they guessed (doable by various methods) Paul’s password(s).

      <

      p>I’m going with the latter.  Google the recent Twitter hacks to see where weak passwords will get you.

    1. Fix soapblox in the short term.

    <

    p>2. Decide the longterm solution when BMG’s current state is secured.

    <

    p>I would make demands for regular copies of backup data being sent to one of the BMG editors, as well as open source coding once this most recent security problem is fixed and everything’s moved to more secure servers.  

  8. Sorry for all caps, but the best long-term solution seems so obvious to me that I need to yell.

    <

    p>Paul’s not trying to be the next Bill Gates.  He has a hobby that makes him a little cash and a whole lot of headaches.  

    <

    p>So why not make SoapBlox a community effort by opening the code and publishing under the GPL or any other similar license?  So everyone who cares to can look at the code, modify it, document it, use it, etc.?

    <

    p>As others have noted, and I tend to agree, Open Source isn’t the solution to all ills.  But it is a perfect fit for this situation.

    <

    p>Releasing the code would help lift the load off of Paul’s shoulders.  Many eyeballs make light work.  It would also allow people to deploy Soapblox on other hosts, either as a backup or permanently.  

    <

    p>I kind of like soapblox (for those complaining about the many clicks to post, have you checked your settings? It’s possible to post in-line on a page, and it’s pretty nice).  But I’m not inclined to give Paul a penny unless I understand if he’s running a for-profit enterprise or a non-profit service.  If the former, I’d be inclined to run away.  If the latter, I don’t understand why he wouldn’t or shouldn’t open the code.

    <

    p>Lots of open source projects have foundations- that’s a reasonable setup.  A 1-man, closed source, sort-of for-profit, blog unreliable hosting service is not.  BlueMassGroup is currently hosted on a machine running Linux and Apache, both of which are open source, and both of which have foundations: Apache and Linux.

    <

    p>If Paul published the code under an open-source license, I’d help support the cost of a programmer, etc., to help him fix things.  Because I know we’d all benefit.  

    • I’ve talked to Paul about this, it’s been planned for a long time. I think the primary reason it hasn’t been is: 1. parts are embarrassingly ugly (programmer pride wants to clean it up first), 2. parts probably do have security holes and he’s been showing the code to a few people to get those cleaned up before it goes fully open.

      <

      p>As for security, it sounds like this attack wasn’t soapblox specific, but merely an attack on some software in the underlying server. I’m not enough of a security nerd to go through and find all those holes and plug them, I’m just lucky that my server isn’t interesting enough to have been broken into yet. Hopefully he’s found a good security nerd to help with his little server farm.

      • to opening up the source, complete with a time frame and scope, and maybe even the license type?  If so, got a link?

      • As a professional programmer, why would I want to give money to someone who won’t open source his code because it is “embarassingly ugly”? That sounds like a bad investment to me.

        <

        p>

        • I submit that the world mostly runs on exceedingly ugly software. When might you incur the expense to make working but ugly software beautiful? When might you be willing to risk breaking ugly but functioning code to improve its unseen appearance?

          • “embarassingly ugly” code is bad not for some aesthetic reason, but because it is seriously flawed. It tends to be hard to maintain, hard to change, and is rife with hidden bugs and security flaws. You don’t rewrite “ugly” code to fix its “appearance” but to fix its deep flaws, and then only when it makes economic sense to do so. Most such software is not even worth fixing.

            <

            p>I agree that the world is full of somewhat or even mostly functioning bad software, but please don’t ask me to contribute money to make more of it.

            <

            p>

Archives