100Mbit Internet

This post contains a little bit of bragging. My internet setup at work is pretty simple, we have two T1 lines (not counting our voip trunk or our DID lines but that’s sort of telephone) going into two different Cisco PIX firewalls and behind those an old Cisco 2600 to do basic routing. Network map One does NATing and port forwarding for our normal internet usage, as well as port forwarding from different IPs for our email and web servers. It’s important that the default route to the net not be the same as the email server as when people get viruses that spam everyone people will stop accepting email from your email server. The other provides vpn access to another office, which also only has a T1 Line. And while T1s are slow, it’s “enough” bandwidth for our business needs and the other office is in the middle of nowhere and can’t do much better. That being said, here in New York City we can do better. A lot better. A T1 usually offers about 1.5Mbit/second for data, I wont cover telephone applications which there are many. That’s fine for surfing the net, watching you tube videos, and email. It is slow for 25 people doing all those things, but more importantly it’s slow for downloading anything of any size. 1 megabyte for example (about a minute of audio, or one large photo – if you knew that I’m sorry to use the comparisons) takes about 6 seconds. 300 megabytes (for example the size of a decent video clip or a Microsoft or Apple security update) takes about half an hour. 700 megs (say the size of a ubuntu install cd – seriously guys no net install cd? I don’t want all your packages.) takes about an hour. t1 weekly You wont see it on this graph as it averages the speed over two hours, but we maxed out our bandwidth quite often. It’s mostly my fault, I download a lot. Our network graphs spike all the time and I can say “oh that was me” for most of them. I probably consume more bandwidth then everyone else here put together. It’s part of my job (and personality) and because I have to share the connection with 20 other people I can’t saturate it for long periods at a time (its rude). At home you probably have about 10Mbit download (700 megs in 10 minutes – but check for yourself) so what slows 20 people down for an hour here would only slow your family or roommate down for 10 minutes at home. Well last week our network graphs automatically adjusted to acomidate a new connection. fiberweekly Have a look at where it says “Maximum” that’s 28 times faster then the other graphs maximum. Technically it could read about 60Mbit a second, that’s the theoretical limit of our firewall. The Pix501 supports up to 60Mbits firewalled, while the Pix 506E does 100Mbit though its firewall it’s busy. What changed was our primary internet connection, we now have a 100Mbit fiber connection from a company called Cogent. They “lit” our building a few years ago but we didn’t have the need or $$ to change connections. It’s now super cheap (~$700 a month – a bargin compared to the ~$400 for a t1) and has proved to be quite relaible. In a few weeks we’re going to move to a Cisco ASA-5505 which will handel firewall, vpn and failover (incase we do loose connection to the internet) drop our remaining t1 line, and steal a few channels off one of the voice T1s for a backup data connection (slow but good enough to keep email flowing). All for less then what we were paying before.

Nice right? Let me put it in perspective. The 700 meg file I can now download in a minute and a half, and when we move to the new hardware it could take 56 seconds. Saving me 59 minutes compared to the origional connection. In actuality we’ll probably never hit full speed as most servers wont pump data at 100Mbit/s nor can you guarentee that you’ll get routed though the net that fast. There’s a noticable speed difference when I pull from california servers compared to new york servers compared to european servers.

My mind is blown. =)

The E-Persons in our lives

Occasionally people come to me and ask about websites.

“Francis, I need help on making website.”
“Well.. what are you up to? ”
“I have an idea of something I’d like to sell online.”

Usually it’s cookies, sometimes it’s t-shirts, a lot of times it it’s services they want to offer. Most of these people have zero programming or html experience. If I told them paypal had an api to allow your site to create invoices and process payments they wouldn’t even understand the non technical part of that sentence. When you get into the advanced parts of how the web works, they glaze over. I usually end up telling them about Shopify which can have you up with professional looking storefront with very little time and effort and as you go you can learn how to make it very pretty and you don’t have to worry about the perils of doing your own web hosting. Personally I’d rather use that $20 a month for my own server slice and spend a lot more time and money on it, but that’s not actually very smart if I wanted to grow a different business that wasn’t web hosting.

The real problem they want to solve isn’t “I want to know how to do a website.” it’s “I want to know how to start a business on the web.” and the core problem there is usually “I want to start a business.” which is usually formed around “I want to have more money”. And wanting more money is not a bad reason to start with, but because you like to do something or make something doesn’t mean starting a business around it is a smart idea. Most of the time your business will take you away from doing those cool things you like and force you to spend all your time doing something else you wont, running the business.

There’s a book I love that will either discourage you or encourage you to start a small business. Either way it will teach you a bit about what you actually have to do to start. I tend to give this book to people who I think it will encourage, as it usually doesn’t take a book to discourage people who would be discouraged. It’s called The E-Myth Revisited: Why Most Small Businesses Don’t Work and What to Do About It by Michael Gerber, and it’s like $5 off amazon used so it’s worth the money, Its worth the read even if you hate the author’s writing style (figure a pound of gold in ten pounds of fluff), and it’s worth your time if you ever thing you might want to go into business. The E-Myth being the Entrepreneurial Myth that a technician can take their idea/product and just start a successful business around it. Most people do that without learning how the rest of the business works.

His book of course is not a blueprint of how to start a small business, grow it, operate it, and sell it, it just tells you that you’ll need one, and what it might look like. I’m rather new at this so it blew me away. The amount of planning and the possibilities you can come up with to make a business work is breathtaking. Its the kind of thing I love. It’s something I’m going to have to write more about. Not to give advice but to share experiences. I can only tell you there’s more to it then you probably know, but in sharing experiences you pool what you learn. =)

-Francis

PS (It’s been too long since I’ve been writing, feels good, but I’m also embarrassed at the writing style I presented above. It’s a silly feeling.)

Windows update fails

I had a problem this week. I actually had the same problem lots of times. It was fustrating as hell.

I had quite a few Windows XP instalations to do, with new employees soon to arrive at work, new laptops arriving with Vista, and some old desktops that needed ‘decrufting’ in their OSes. I have a slipstreamed windows xp cd with sp2 that I like to use. I haven’t bothered updating it to sp3 because I haven’t made the time and for a while I didn’t think it was worth it. SP3 seems more for microsoft then it does for the users. I do have a new xp disk with ie7 and a few other updates, but I haven’t tested it so much. So I tried it out and it seemed ok but I hit a problem.

Windows Update would fail to install every single update after sp3 was installed.

And once you hit Microsoft Update (which you should) it would fail to install the office updates too. No error codes, no event log messages (well maybe but I didn’t look too closely) nothing useful. I figured it was my new disk so despite spending all the time reinstalling windows (my disk does make it easier, setup the partitions and walk away for 20 minutes and you have windows waiting for you when you get back) I tried with my older trusted disk. Same problems.

It took me a while to figure out it was sp3.

Well I have a fix. Run these commands.

::http://support.microsoft.com/kb/943144 for details use on 32bit winxp only
net stop wuauserv
regsvr32 %windir%\system32\wups2.dll
net start wuauserv

Put that in a batch file or just run it and it will re-register the new sp3 windows update dll file and all your problems will go away. So in an effort to take pride in my work I decided to fix it twice. Once for the problem and again for the cause of the problem. According to the knowledge base article I found to fix it, if you update windows update (which is common after a reinstall) and don’t restart before installing sp3 then sp3 will ignore the updated windows update and there will be a version missmatch when you try to update. So restart early and often when installing updates. I haven’t tested it but in the back of my head there’s a voice saing “I think other updates might cause this problem too!” so maybe even something other then sp3 can cause this, but I don’t feel like collecting proof.

That would involve “fixing it” 3 times and twice is enough for me today.

-Francis

UPDATE: So far this has only applied to new installs for me, but if you’re worried you wont hurt anything by running those commands.

Belkin good Comp USA bad

I almost got into a fight with a guy at Comp USA. Well that’s a lie. I just didn’t want to buy the $180 fancy Belkin router with the lcd display and curvy exterior.

“Does it have better range?”
“Yes”
“Then the linksys? or compared to the cheeper belkin”
(Which I might add I did buy, it’s got an amazingly simple setup and a very strait forward instructions that I would trust my mother with. Never mind I skipped all that because I had to set it up to be transplanted to another location later.)
“Both, these are like around the house, this one is beyond the walls.”
“Does it have more millawats or something? Does it say?”
“This is the one everyone is buying, we’ve sold like 15 today.”
“It’s too expensive” I say looking at the cheeper Belkin.
“Look, it’s only $80 more, its the one you want.”
“It’s not my money” Which was only a half lie. I had stopped making eye contact with the guy and continued to look for some indication that the 802.11N spec was different between the different routers and somehow gave it better range.
“Do you work for a company or something?”
“Yes”
and then he walked away.

I went for the one with 3 antennas, Belkin makes OK routers and linksys’s 2 antennas hasn’t earned any respect with me in the wireless range department and I wasn’t spending that extra 80 bucks.

During check out another guy runs up to my cashier.
“Hey hey! don’t forget to put in your code! Is that one of those!?” he says pointing to the $180 routers I didn’t want.
“No” she replies.
“Well if you sell any of those you get $5 for each one. Five dollars!”

I hate compusa.

How do I phrase this?

We’re leaving one of our ISP’s here at work. And I was asked to take care of it. We’re still on contract but they broke it long long ago.
I’ll have logs of downed an unacceptably high latency connections and all the excuses they’ve given us over the months. But I need to start the ball rolling by writing a letter. This is what I have so far..

To: Towerstream
Subject: Termination of Service
John,
We’d like to leave your shitty fucking service Immediately.

You lied to us.

Love,
Francis

The spam we get is of biblical proportions

Spam is a problem, and at my day job it seems to be an ever increasing one. We have two main methods of blocking spam, but lets start at the beginning.

Our server accepts an email to a valid address. (And tells the server to bugger off if there’s no account here by that name.) Its scanned for virus’s and email client exploits, as well as blocking encrypted archives, scanning inside archives and scanning for macros in Microsoft Office documents. This stops a great deal of mail but not the majority of it. Next it’s put through a gauntlet of checks making sure it’s not spam. There are 12 modules that check the message, a collection of black list and white lists (sorta Greylisting) that can reject, accept, or pass on a message and send it down the chain. We have a large collection of manually added white listed addresses and automatically entered whit listed addresses. The mailsystem scans both incoming and outgoing mail for both content (bayesian) and email addresses. So if you email someone the reply won’t get caught up as spam.

We also employ real time black lists (or RBLs), which checks the ip and hostname of the server sending the messages (does a dns lookup) and checks it against several (we’re using 3 different sources right now) databases of ips that flag for spam, and abuse. These services have vast networks that just receive spam and virus attacks and log and identify them so other people can block them.

We have a spam cache, we don’t send the spam to our users junk folders, most of them don’t want to see it. out of the past 12,746 messages, 7,246 of them were caught with the RBL, and 5220 of them with the bayesian content filtering. And yet, still many get through. We get about 2000 spam a day caught and I think that’s only about 95%. Work that out over around 70 some odd users and that’s about 30 spam per person per day. Which sounds about average.

But biblical?

Well

SUBJECT: And Saul as I have gone out, of Jephunneh.  And thy God, surely there
MESSAGE: to meet with water: Which the reward for thou, was under the land slew

SUBJECT: certain woman when the became a word shall be kept the flock,
MESSAGE: vessels: thereof, three hundred made before thee, and took Ishmael

SUBJECT:no; more, to his days and the great price.  But thou
MESSAGE: promised to the posts thereof, are all the men and when they have left

SUBJECT:Jekamiah, and is thou in peace from Assyria; have access
MESSAGE: shall the son that remain, in it If not on the rough wind into

That’s not even a little of it. Eventually they started coming it with the quote and image spam. Pictures with text advertising drugstores and stock tips. The first messages were to soften up our spam filter to let the other ones though. Go figure.

-Francis

PS I get about 150 – 200 messages a day over a handful of accounts and my osx Mail app misses about.. 20 or so. While it’s % caught is not very high it’s a lot lower volume then at my 9-5.

SubEthaEdit

SubEthaEdit is a really nifty text editor with sytax highlighting. What does that mean?

<?php
require_once(‘core/core.php’);
//header smarty
$smarty->assign(‘title’,‘Homepage’);
$smarty->assign(‘location’,‘home’);
//end header smarty
$smarty->assign(‘mainwelcome’, ‘Please choose a component to use.’);
$smarty->assign(‘maincomponents’, array(
        array(‘url’ => ‘dataimport/’, ‘title’ => ‘Data Import’, ‘desc’=>‘This is where you manage importing datafiles.’),
        array(‘url’ => ‘export/’, ‘title’ => ‘Export’, ‘desc’=>‘Here you can export data in predefined formats.’),
        array(‘url’ => ‘datavalidation/’, ‘title’ => ‘Data Validate’, ‘desc’=>‘Here you can validate data.’),
        ));
$smarty->display(‘indexmain.tpl’);
?>

(That’s some simple php code I use in a project.)

Each element of the sytax is highlighted and it really assists in editing. But syntax highlighting is nothing new. SubEthaEdit brings in network sharing of a document and other nifty things you can read about everything on your own.

I bring this all up because I added the extension tpl to the smarty-html mode. You can download it here. I did it because I was sick of all my .tpl files not automaticly highlighting. (.tpl is smarty’s default file extension for template files.) And I put it here to show off SubEthaEdit and it’s html export and because at least one person I know will find it useful.

I learned of one extra feature from SubEthaEdit today called block editing. There’s a movie showing it off and may I say, god dammed that’s really cool.

-Francis

Backuping hard drives

I was writing an email to my mother about different backup methods (I kid you not) and I thought some of it was useful. And before anyone says hardlinks or rsync, I know, I know. It’s just there isn’t any really good way to do that on windows and I’ll take incremental drive images over 100,000 file hardlinks and rsyncs. I swear I’m going to run out of innodes before I run out of hard drive space.

They’re opening up a commerce bank down the block.

I’m happy about the commerce bank being built down the block. The private parking lot was nice and all, and I did enjoy looking at the converted pig box that used to serve as a delivery.. vehicle, but I think the bank will be a lot more useful. I’ll be able to walk down the block to count my change for free. (oh and do banking and things)

I was looking for a link on the penny arcades but I found nothing that just showed what it was (it counts the coins) but I did find out that they’re easy to rip off. Apparently a bunch of them have been robbed. I just hope when I go to rob a bank my first instinct is to go for the spare change. I just hope they don’t have this guy working security. I took that friday night.

I know it’s been a while since I’ve posted (so sue me, cancel your subscription), I’ve been working mostly, taking breaks for trips to beaches and birthday parties. (Matt, I found your dream girl by the way.) And occasionally the friday night game of pool with Jason.

I don’t know if I ever mentioned my credit card fraud problem, I got most of my money back the other day (all but 4 bucks says quickbooks) and I’m sitting pretty about it. Now I just need to solve my no credit problem (more on that another day).

This is looking pretty attractive. There were talks with Tivo and netflix for a home “download” box where you could download your movies overnight and watch them as much as you want. Their always worried that it could be used for evil, like to copy the movies or something, but I can do that with dvds. I just did a quick Froogle Search I found out it runs around $600 bucks. No longer as attractive.

Don’t get me started on $800 dollars in one day, if you’ve heard my rant then you know I want my money back. Unfortunately the people who need to give my money back haven’t heard it yet. You know who you are! You best be better getting with the check writing you people you.
(I thank you Hawkeye Pierce)

On that note I watched The first DVD of Season 1 of Mash today. It’s a lot better without Devry Commercials or a laugh track. I love this show, every eyeful.

Good night folks, I’ll be here all week.

-Francis

PS Look for new Audioslave, Cold Play, and The White Stripes

Alive and kicking.

I’m tired.

But its well deserved.

Sort of.

So I got up today and went to work. I took a cab because I was late. I had a 3:30 appointment that got rescheduled for 11:30 but some stroke of luck I had arrived at 11:10 – early. Spring and Greenwich (near the west side drive) isn’t that far from brooklyn. Someone let me in the building and I took the old freight elevator upstairs. The entire office building used to be some industrial something or other and now is chic (but I use the term loosely) loft office space.

The best thing about this office though isn’t the sky lights on the top floor (though they are nice) or the wooden support beams (which are left unpainted) but the fact that before you reach a single office door there is a washroom. Its nice too, got fancy doors on the stalls and two big sinks on either side of a large mirror. The first time I saw the office it was to talk about getting a new job. I was really nervous and it was hot outside and my bag was heavy. I arrived and I felt like I was in greece. I was only just announced and I was given a chance to wash my face, my hands and cool off. The greeks they didn’t even want to know your name until after you were washed, rested and fed. It was quite refreshing.

Anyway I was late the meeting was really supposed to be at 11 not 11:30 but at least the other guy was later, he didn’t get there until 10 minutes after I did.

And instead of the “This kid doesn’t know what he’s doing, we need to find someone else to trust the future of this company in.” that I was expecting I got a “well you’ve got a start but this will help you” and he proceeded to help me lay out the beginnings on the entire project. Man, I need to take a course in planing or something, I learned a lot today.

It was good. Class was Boring! But hey at least I learned a few things. I also found my friend Nina on tv (not for real), and England keeps coming up and that turns to Ellie in my mind as she’s the only british person I know (Sorry Mike you don’t count. I guess its time you became a citizen or something.) I’ve got about 100… no 1000, or maybe even a 10000 pictures to upload from Brian’s party. I also setup 2 new galleries for other people and moved all my friends and respected photographers galleries into a single album. None of the links changed so no worries.

Brian’s Party

I’ll have other people’s photos of the event up soon.

-Francis