Please go to www.channeldynamics.net for all Small-Tree related products and enquiries.

Telephone
0844 800 0540
International: +44 1753 502680
Support: 0844 800 0542

Interesting and detailes article on the Fibre Channel Protocol from the EnterpriseStorageForum.com

Moved this from the contact page:

Luke Halls Sep 19th, 2007 at 2:55 pm Edit

We are currently looking at installing a SAN in our production company and i am looking at getting some idea of how much it would cost to use 10GbE. We have 8 macs that we would like to share about 10TB of storage between. I would also like to have an idea of the benefits / drawbacks of using 10gbE compared with fibre. Is this something you could help us with? if not could you point us in the right direction?

1st response from Small Tree:
Hi Luke,

The original idea behind the SAN scenario is that the data path and the meta-data path would be separated allowing more scalability. In theory, that works and I think for installations of your size, it probably works fine. As installations get larger, the notion of scaling the meta-data network and traffic gets tough. There are SAN solutions like Lustre that are trying to solve that. We’ll see.

10Gb to a server would be a great thing, except that with 8 clients, you’re looking at 80Gb/sec of bandwidth to and from the server. I don’t think you’ll get that through anything within your budget.

We’d need to understand what your clients are trying to do (edit SD video off a shared storage location)? and then we could figure out whether Gigabit or 10Gb could meet that requirement. I have customers today serving 4 or 5 stations happily with Gigabit off a server, no special SAN software required.

2nd response from Small Tree:
One really needs to understand the requirements of the customer before we can say that 10Gb ethernet is better (or worse) than a fiber channel install.

The theory behind fiber channel is that each machine has a direct connection to the data path. That’s a good thing. However in order for that to work, one must have enough fiber channel ports (expensive) and some kind of volume sharing or cluster fileserver type software (typically $1000 per station). Additionally, your clustered filesystem will also need a metadata server. The (perhaps slight) advantage of a meta-data server over a normal fileserver is that in a regular file server situation, the server has to be able to handle the entire data path requirement. 8 systems (as in this case) might demand 1.2GBytes per second (assuming roughly one HD stream per system).

The question is whether it’s cheaper/easier/more reliable to buy a really fast server that has a big data path or spend that same money on FC switch ports, cluster software and a meta-data server. Given the feedback and performance data so far, I’m still leaning towards servers.

Now here’s the rub: Customers are always willing to pay a little less to get a little less. Money is generally tight and people like find ways to trim spending. When I talk to customers and they hear the price tag for HD bandwidth to every station all the time, they generally tell me that they don’t need that just yet. Usually, it turns out they need 3 or 4 compressed streams of SD with a total bandwidth of less than 50Mbits/sec.

Given this scenario, I advise most customers with less than 8 clients to consider a server with a lot of memory and striped storage. This generally works pretty well, especially with the latest final cut pro (6?). It does a much better job of buffering than in the past and with 10.5, I expect that solution to get even better. It’s extremely inexpensive compared to any sort of cluster or volume sharing set up with fiber channel.

8 mbps (megabits per second) = 1 MB/sec (megabytes per second)

Some definitions for the confused:

• http://wiki.answers.com/Q/What_are_megabytes_and_megabits
• http://en.wikipedia.org/wiki/Megabit
• http://www.netbook.cs.purdue.edu/othrpags/qanda188.htm
• http://www.wisegeek.com/what-is-mbps.htm

Hope this helps..

Given a TCP/IP connection one can expect to achieve over 6Gbit/sec from Small Tree cards which in my view is very nice thank you very much..

If you use a protocol such as AFP then you will be lucky to achieve 1Gbit/sec.

When to choose copper:

• Cost Sensitive
• Short distance between servers

Some info from wikipedia on copper technology

10GBASE-CX4 Technology.

It is designed to work up to a distance of 15 m (49 feet). This technology has the lowest cost per port of all 10Gb interconnects, at the expense of range.

10GBASE-T

A standard released in 2006 to provide 10 gigabit/second connections over conventional unshielded or shielded twisted pair cables, over distances up to 100 m.^[2]

Performance Question:

“Can I really achieve 10G/second between two servers connected with 10Gig Ethernet adapters?”

PCI-X

Majority of people are asking us for pci-x, must be that hardware refreshing does not happen as much as the industry would like to think.

Found a related article from 2004:

“Furthermore, PCI-X is not yet an end-of-life technology. The graphic below shows there still is some life in PCI-X beyond what is offered in Power Mac G5 and other PCI-X systems.” http://www.it-enquirer.com/main/ite/more/pci_xpci_express/

An interesting article in the Guardian …

http://www.guardian.co.uk/technology/2007/may/31/comment.guardianweeklytechnologysection1

Could fibre techology really become as cheap as copper?