Thursday, September 28, 2006

Things are Busy, Check Out Lubbock's Incubator Project

Things here have been extremely busy over the past few weeks. We are still getting used to our new location, the bright colors that our group picked out for their offices, and everything else that goes along with a move (including our bad plumbing, and a condemned heating system).

In the midst of all of the changes, I got a call from Bonner Bennett, the Chairman and Director of the Technology Research Association Consortium (TRAC). Just when we thought there were enough acronyms ..... .

Seems that TRAC has secured some funding, and in coordination with the local SBDC, the Texas Workforce Commission, and [maybe?] Texas Tech. My details were a bit fuzzy (and yes, I'm listed as a member of TRAC's Consultant Board). John Smothers (the owner of TG Trees) is directing the incubator, which is located somewhere in Lubbock (I believe at the Science Spectrum building).

So, anyone in the area interested in taking their technology to an incubator can start with one of the above persons.

My advice: Protect yourself at all costs, including patents / copyrights / trademarks, and don't disclose more than you need to at any point-in-time. When you go and talk with them, remember that you're interviewing them, as much as they're interviewing you. But then, my advice works for everyone, not just the Lubbock Incubator project.

Monday, September 18, 2006

NTS Communications in Lubbock Tries CoLocation

After the NTS Communications sales reps ran into some problems helping me with a quote for T1's, I inquired about colocation at their NOC in Lubbock. I've looked at putting servers at ServerBeach, Kattare, and RackSpace in the past, so I was familiar with the pricing and the market. After all, it may be easier to put the servers at a local facility, rather than hosting them at our location.

Daryl (or Darrel or Darrell) called me back and asked me what I wanted. The conversation went about like this (D is him, M is me):

D: "What can I help you with?"
M: "I'm interested in getting some information about colocation at your NOC"
D: "We do that. What is it you want to do?"
M: "We are developing some web applications, and rather than bringing in the data lines, I'm looking at options for hosting servers elsewhere."
D: "We do that. What are you going to do?"
M: "Um ... we are going to host web applications. Probably 2 servers."
D: "We do that. Do you know what U is?"
M: "Yes."
[Note: A U is a measure of height in rack space. It's not very tall, but is 19" wide, and quite deep.]
D: "We charge $100 per U."
M: "Ok."
D: "How many U's do you need?"
M: "I suppose we'll need two U's, for two different 1 U server's."
D: "Well, we can do that. We have a Halon system, redundant backup generators, and physical access, and everything you'd get at any other data center. You just need to provide the servers, and UPS's."
M: "UPS's?"
D: "Yeap. That's for your protection."
[Note: Most data centers provide UPS's. In fact, this is the only one I know of that doesn't.]
M: "Well, UPS's are usually taller than 1U [at least consumer-grade UPS's are]. Do they take up 2 U's, or can they go on the floor?"
D: "They take up 2 U's, unless you get the more expensive kind. So, that's 4 U's that you are up to."
M: "Ok. How much?"
D: "Well, it depends on what you need. If you'll tell me exactly what you need, I can get you a custom quote."
M: "Um ..... ok."
M: "I guess that's it. 4 U's of space for 2 1U servers, and 1 or 2 UPS's"
D: "Well, I can give you one cable to plug in. You'll need more than one. Do you have a router, a switch, or a firewall?"
M: "I thought you said I just needed the servers. What other equipment do I need to provide?"
D: "Well, you'll need a firewall / router, which we can put on the same shelf."
M: "Um, ok. What else?"
D: "How much bandwidth do you need a month? We can go up to 100Mbps"
[Note: The question and the unit of measure are totally different. He basically asked how tall I was, then went and computed the sun's gravitational force on me.]
M: "Um, just around 100-200GB / month"
D: "Are you serious?"
M: "Yes."
D: "Well, what you do with the bandwidth is up to you. Will 3Mbps be enough?"
M: "Yes. 1.5 would probably be fine."
D: "Ok. Hang on."
D: "Well, that will cost you right at $1,000 per month."
M: "Wow. That's a lot."
D: "Well, we can bring out two bonded T1's to your location for $800 / month"

Can you tell which product they are in the business of selling?

[Note: Colocation of 2 U's from Kattare with the necessary bandwidth, etc... would cost me about $230 / month. Also, for $1000 / mo, I can bring in an OC3 or DS-3, and get 3Mbps, up to 45Mbps. I can also get 3 bonded T1's from SpeakEasy for $989, which would be 4.5Mbps].

Sunday, September 17, 2006

NTS Communications in Lubbock

Last week, we had a need to replace our DSL / phone carrier, SBC / at&t. You can read about the fiasco here, but they didn't earn any high marks from their quality of service, or their response times. Their techs were unwilling to do more than the minimum to get the job done.

At the same time, I had a meeting with some members of a local children's clinic, and we will be moving forward to help them with their local connectivity, network systems, etc... Which means that we will be hosting their systems at our location, and will need more bandwidth than what a single DSL line will provide.

I first called SpeakEasy, because they were having a special on their T1 lines, at $389 / month, including a $599 router and guaranteed uptimes. After I spoke with their sales rep twice, he re-ran our address, and decided that the $389 / month special was not in a servicable location for us, so the cost would be $599 / month. Yikes.

I ended up calling all over town, including New Edge Networks (who can offer a T1 for $499 / mo, or SDSL at T1 speeds for $199 / mo), SBC / at&t (who can offer a T1 for $599 / mo, or another DSL line for $79 / month). I also called SuddenLink Communications, the local Cox Communications company, who offer no Service Level Agreement (SLA) on their lines, and want $159 / month for a 3M down / 512K down Cable Modem with a 3-year agreement.

As a last ditch effort, I called our local NTS Communications, who passed me off to two different sales reps to handle my questions about SDSL, Fiber, and T1 connections. After some initial confusion, I found out that they don't really have anything to compete with SDSL in my location, and Fiber is not going to be a long-time coming to our location (which would offer 3M up and 3M down, about two T1's). They are running a T1 special (1.5M up and 1.5M down), for $300 / month on a 2-year commitment. However, in the 12 months that they've had the special, neither sales person had sold one, so they could not answer any of my questions.

That's when I got Daryl - an engineer in their Network Operations Center (NOC).

Saturday, September 16, 2006

SBC - Round Four

I got a random fax on Wednesday from SBC, which was three pages long. The first was a cover page. The second was a one line note:

Please call SBC at 866-583-2494 regarding your dsl Trouble Report XXXXX28. You can refer to your DSK telephone # when you call in.

The third page was the trialing end of a URL. Very strange.

So I called, and got a nice gentleman on the phone who explained that the tech that they had sent out on Monday had left the ticket open, and they were curious if they still needed to send someone out. However, they didn't have anyone available for Wednesday, or Thursday. When on Friday would be a good time for me? I hung up.

[Note: If I had not called and called and called, and hounded SBC / at&t about their service and installation, we would have been down more than 7 days to move a DSL phone line. Ridiculous.]

Friday, September 15, 2006

SBC - Round Three

Saturday morning, and there is no connectivity. Nothing. Blinking green lights. I spent three more hours on the phone with level one, and then level two tech support. They decided to schedule to send someone on Monday morning.

On Monday morning, at about 8:15am, an SBC tech walked in off the street, and plugged in his laptop, and a new modem. He was there for three hours before rolling up the cords and telling me that they found the problem, and it would be end-of-day before it was up and running.

What could possibly be the problem? He explained that our sticky-static routes did not make the jump from one Redbag to the other Redbag. A Redbag, as it turns out, is what at&t uses for a router (anyone got a link?). End of day to change a route?

I immediately called Sam, and explained that this was unacceptable. We were down on Thursday, Friday, Saturday, Sunday, and now all day Monday? For a 15 block move? Sam called Deon, and a few hours later, Deon and I were discussing sticky-static IP addresses, and the fact that the correct route was, in fact, in the correct Redbag.

We desperately needed to be online, so he suggested that we punt, and just issue new static-IP addresses. He disappeared for about two hours, then called back to let me know to restart the 2wire modem. About three minutes later, we had service. Amazing!

I called Sam after I got off the phone with Deon, and asked him if I was going to be expected to pay for the move, and the installation charges. He agreed that didn't seem like the correct thing, and would call back with someone from billing to take care of the situation.

No word yet from him.

Thursday, September 14, 2006

SBC - Round Two

At 8am on Friday morning, I got a call from the installation tech at SBC / at&t. We were in the middle of our physical move, so we setup the phone / DSL install at 10am. He seemed to be pleasant enough, and I showed him where to install the physical jack and the new wiring. Turns out, he had both of our tickets, the DSL and the phone, on his docket.

After about an hour, he told me that he was done, and pointed at the jack. I smiled, and calmly asked if I could plug my modem in. After all, it should work, right? He agreed, and I plugged in the DSL modem that we had brought over with us from our old office.

The top light went green. There was power.
The bottom light went green. There was local network.
The middle light blinked green. And it blinked. And it blinked.

While we watched it blink, the tech informed me that our brand of modem, 2wire, was not very good, and that tech support could dispatch someone to get us a new modem. He turned around, collected his ladder, and walked out the door. Talk about service.

I immediately called Sam, at at&t, who put me in contact with Deon again. Deon and I proceeded to spend about three hours off and on the phone, and he could not replicate the problem. At all. At about 2pm, he called the DSL group, and they reminded him that the service had not yet been provisioned, and it wouldn't actually be provisioned until end-of-day on Friday. So, we'd be down the entire day!

He did tell me that if it wasn't up by 6pm, to give tech support a call.

[As a note to our loyal readers: I can throw a soda can and hit the SBC repair facility, it's across the street on 40th Street and Frankford.]

Amazingly, 8pm came around, and ... nothing. Blinking green lights.

Wednesday, September 13, 2006

SBC - Round One

On Tuesday, September 5, 2006, I called our phone and DSL provider, which used to be SBC / Yahoo, and is now at&t / Yahoo. I was calling to setup a date and time to move the phone and DSL to our new location, just about 15 blocks from where we were.

After searching around their website for a direct phone number, I grabbed a phone book and looked up SBC, and gave that number a call. I had been on hold for about 18 minutes when Mary got on the line. I explained our upcoming move, and she explained that we would need two different technicians to actually perform the move. One for the DSL. One for the actual phone. We initially setup the phone on Thursday, and the DSL on Friday, but Mary decided that we could do the entire setup on Friday morning. Minimal downtime! Perfecto!

Imagine my surprise on Thursday morning around 6am, when my cell phone rang from our monitoring service to tell me that our DSL line was down. It was down until 8am, when I walked into the office, and got on the phone with technician one.

After about 35 minutes of phone work, technician one walked me through getting to my modem's homepage (although she didn't know the difference between a homepage and a subnet). Time to transfer me to level two technical support. George. Technician two.

George spends roughly 30 seconds looking at my record, and decides that the DSL group took down the circuit 24 hours early, and we would be down an extra 24 hours. I explained to him that I was TICKED, and I needed someone to yell at. George gave me the number to the executive office, so I could speak with someone there. I got Sam.

Sam's phone number is: 925-973-7751 .... if anyone needs it.

Sam was nice enough to put me in touch with Deon, and Deon and I spoke about the fact that George was completely wrong. In fact, Deon claimed that our circuit was actually the problem, and was down. Tech support had completely screwed up and took the easy path out (thanks again George!) By 3pm he had our circuit back up and running, and apologized for the problems. But we were not yet out of the danger zone...

Saturday, September 09, 2006

Site24x7 appears to be very slow

Just a quick update to my review / post about Site24x7. One of the sites that I have configured Site24x7 to monitor, every 15 minutes, is our IP address, and our own in-house / Intranet webserver. On Friday morning (yesterday), at around 8am CST, I turned off the firewall and modem, which should have triggered an SMS message within 15 minutes.

At 10:38am, I received the message. I'm not sure if that is Sprint's problem, or Site24x7's problem, but it certainly could have been a problem with a business-critical web site.

A Long, But Short, Move

It's official. We're two items away from being fully relocated to our new location, at 5760 40th Street, Suite C, Lubbock, TX, 79407. Courtesy of Lamberts Properties, including Tristan and Mike. The last two remaining items are a heavy-bag and speed-bag combination work-out equipment, and an all-in-one gym set, both of which are scheduled to be moved tomorrow (Sunday) AM.

Last I had checked with our old landlord, the D'Cruz's (Neurologists in Lubbock running Lubbock Neurology), she already begun bringing in contractors while we were moving out on Friday.

This particular move was a very quick move; it took us less than 4 hours to actually move everything out of the old location and to the new location. However, I've spent a total of SIX hours on the phone with at&t (the new Southwestern Bell Company) trying to get our Digital Subscriber Line (DSL) services turned back on. I'll have more about their crummy customer service in a future post, but we've already started looking at alternatives to their service.

Wednesday, September 06, 2006

Site24x7 Web Review

We manage many different websites for ourselves and our customers, and we need to make sure that they are consistently up and running. Over the years, we have used many different solutions, including commercial off-the-shelf software and custom developed software.

Everyone who owns, runs, or maintains a website needs to make sure that it is up and running.

Recently, the developers and business leaders behind the Zoho suite of software began Beta testing their new product, site24x7. We have been testing their product for about 4 days.

After signing up at Site24x7, you define any number of sites by clicking on "Monitor a Website", or you can monitor a transaction (I have not used this capability, because transactions would need to be read-only on most of our websites). On each website, you enter a URL, a frequency to monitor, and then can choose some "Advanced Options".

Advanced Options include notification when certain words are, or are not, present. It also includes when the content changes more than some percentage (I did not test this, and I'm not sure Site24x7 considers the initial content) and when the response time is too slow. You can also define one more contacts to email in the case that any of the events occur. Site24x7 can also handle POST or GET requests, and can handle Basic authentication (I would love to see it support FORM based authentication).

Contacts can be either email or SMS based, though they set the limit at 10 SMS messages per month per web site. I'm not exactly sure what happens when you should get the 11th message (if they just do not send it, that could be a major problem for some web masters).

Each site that is monitored has a "report" page allowing you to see a graph of response times, up-time and down-time percentages including historical and daily averages.

A few other sites have had good things to say about Site24x7, including a review here and Download Squad.

Of course, it's Beta software, and we've found a few flaws with it. I took down a site for about 30 minutes, and was never notified. Oops.

Saturday, September 02, 2006

Small Business Lesson One: Use Backups

Consider this: The average desktop-grade computer fails once every three (3) years.

Hard drives fail. Power supplies fail. Fans inside the machine will break down over time (and a dirty environment will only accelerate this process), causing CPU's and motherboards to overheat and fail. High powered video cards now come with fans and heat sinks, and they can fail too.

Depending on your requirements, your backup strategy could be CD or DVD media, an external hard drive, a dedicated backup storage solution, or tape backups. Your backup strategy could be local (at which point you need to make sure that you occassionally take backups offsite, in case the building burns down, etc...), or remote.

Backing up your data is only the first part of your backup strategy. The more important step is the restoration of your backups. Without the ability to restore backups, backups are useless. Make sure that restoration works.

SOHO operators should take a look at Carbonite backup. For a flat-rate ($5 / mo), and available for purchase online or at Staples, Carbonite will backup and restore an unlimited amount of data. It is not very fast, but it does work. On a recent trial that I did of the software, it took almost four days to backup about 20GB of data and about the same amount of time to restore it.

Another possibility for SOHO operators is JungleDisk, a great, free utility that is built on Amazon.com's S3 data service. Keep in mind that Amazon's S3 data service is not free (but quite affordable).

Small businesses (under 20 - 30 employees) should look at either an external large hard drive or a single tape drive. One good option is a Western Digital's NetCenter Network Drive. This $400 solution comes with EMC backup software and should be able to backup a small network with no problem. This product could be used in combination with Carbonite or JungleDisk (backing up the actual backups to their remote service) for a complete backup solution.

Larger backup needs can be solved using backup servers and tape auto-loaders. A custom server with six (6) redundant, large hard drives can be assembled for under $2,000. One possibility would be to backup the network onto this type of server, and then store these backups on a tape auto-loader. One of the best tape auto-loaders on the market is the Exabyte VXA-172 PacketLoader, which can handle more than 1TB of data without having to switch tapes.

For those who are interested, a great resource for advanced storage needs is StorageMojo.