Networks Spanning Multiple Sites


Problem Definition

Our small nonprofit has multiple sites that need access to the centralized resources. The two remote sites currently use multiple dedicated site-to-site T1s (2 T1s at 1.5mbps) that are both expensive and slow. Each site has at least 20 users. Our organization is about to open a third remote site and needs to enable access. The most common issue amongst the remote sites is network speed and reliability.

The organization uses multiple services ranging from internet-based hosted email, a few cloud applications, a few network based applications, and a series of centralized file shares. Traffic analysis shows that for each site, there is a 75/25 split of usage with ¾ of the traffic being internet, and ¼ of the traffic being WAN. Our servers are located in one main office, and internet service is currently being provided through that office to the remote sites over their dedicated links. An outage in the main office also disables all remote offices completely.

As for costs, the organization already owns all the equipment needed for this current system to work at three of the sites, but would need to purchase service and equipment for the new site. Each individual T1 (2 per extant site, and 2 for internet service at the main office) costs approximately $700 per month. Assuming that we bring the third remote site online at the beginning of a fiscal year, with an initial equipment purchase of around $3,000, your first year costs to continue to provide the same caliber of service is ($1400 x 4) * 12 + $3,000 or $70,200.

Available Options


Obviously, the first available option is discussed above. It’s a known quantity, works (albeit with some degree of user dissatisfaction), is the most secure, and requires no major capital expenditure.


Another option would be an MPLS network over fiber or fixed point wireless. The fiber option is going to end up with about the same or greater monthly costs, and there will be a possible capital expense regarding routers at each endpoint.


The third option is to take advantage of improvements in consumer technology. A business class cable modem, while unable to provide dedicated bandwidth, can offer generally far faster service at a huge discount. Cable modem service with dedicated IPs and 50 mbps download and 10 mbps upload run about $250 per month. There are a few capital expenses in this option as well – each site would now have an internet connection instead of a private link and would need a firewall, which in turn means there must be some way for the sites to access those centralized data sources. Also, while you could still route domain controller functions in a few options, it would probably make more sense to provide local access to that sort of data so that at least internet resources are fully accessible during an outage in the main office area.

For simplicity’s sake, let’s overestimate the capital outlay for each site to $10,000 dollars. This means our first year’s cost would be ($250 x 4) x 12 + $40,000 or $52,000 a year (also, known as just under $20,000 less than the extant system with a massive difference in capital expense, meaning Year 2 would cost about $12,000 vs. $67,200).

Even if the initial capital investment needs aggressive replacement (every 3 years versus every 4 or 5), this option is a huge money saver. Also, given full intersite utilization, there is about equal speed (say about 9 mbps allowing for a bit of overhead split 3 ways, or 3mbps). With most companies, the utilization would never be this aggressive, and regardless would only apply to the 25% of traffic hitting the central servers in the main office. The 75% of traffic going to the public internet would be radically faster.


In implementing Option 3, we utilized SonicWALL firewalls and point-to-point VPN links along with local backup domain controllers. We experienced speed complaints at remote sites once every three months in extreme situations, as opposed to near daily prior. Outages were cut by about 50% in the first year.  As our costs were so much lower, we were able to offer better phone service to these sites as well as backup internet connections at two of the sites by Year 2, bringing our overall downtime to 10% of original, or about an hour a year across all sites. Even with this radical increase in quality and reliability, the costs have decreased significantly.