by wuhaijia

Slides
122 slides

FFS:一种基于网络的PB级云存储系统.pdf

Published Oct 3, 2013 in Research
Direct Link :

FFS:一种基于网络的PB级云存储系统.pdf... Read more

Read less


Comments

comments powered by Disqus

Presentation Slides & Transcript

Presentation Slides & Transcript

TP338.8 FFS PB

I Jim Gray 18 I/O PB Formicary File System FFS PC (1) (2) (3) (4) FFS FFS FFS 17 PC FFS 105MB/s 49.4MB/s (5) FFS FTP FFS FTP

IIAbstract Jim Gray, a Turing Award winner, has dr awn an empirical law: Under network environment, the volume of data, which wi ll be generated durin g the next 18 months, equals to the total volume of data generated throughout history. The explosive growth of information resources make hard demands to st orage systems' vloume, expansibility, data reliability, I/O performance, storage management, power consumption and total system cost. Most of traditional mass-storage systems are high-cost, hard to manage, and difficult to upgrade. So we provide a new PB-Level Cloud-Storage System named Formicary File System (FFS), which draw on the idea of Util ity-Storage and Storage-Virtualization. FFS is built from many inexpensive commodity com ponents that often fail, it has parallel expansibility, it can switch failure and renew data automatically, and it can balance work-load and data-load dynamically. Through the technology of vi rtualization, it can provide the aggregate storage resource to th e client in the form of a common disk. The major contributions are outlined as follows: (1) The development history of traditional ma ss-storage systems is analyzed, and their architectures and key-technics are summarized. (2) The well-known cloud-storage systems bot h domestic and overseas are surveyed and crosswise comparied, and their rela tive merits and applicable scopes are summarized. (3) Several releative technics are researched, including the techinc of storage virtualization, the mechanism of data re liability, the self-management method of storage cluster, and the ROA (Resource-Oriented-Archi tecture) based software design method for cloud storage system. (4) We designed and implemented FFS, and in this paper, we set forth the architecture and key-technic of FFS, and make a test to wards FFS. The test results show that at the experiment condition of a 17-PC- built cluster, the maximum aggregate read-rate can reach to 105MB/s, as well as the maximum aggregate write-rate can reach to 49.4MB/s, and dur ing the test, FFS behaves well at load balance. (5) We provide an application case, in which FFS is used as the base storage system for the FTP service in a campus network. The application effect ve rifies that, using FFS can promote the concurrent performance and expansibility of the FTP service, and can reduce the setting up cost largely. Keywords: FFS; Mass-Storage; Cloud-Storage; Utili ty-Storage; Storage Virtualization;

III .............................................................................................................................. ........ IAbstract ...................................................................................................................... ........... II .............................................................................................................................. ..... III .............................................................................................................................. .... V .............................................................................................................................. VIII 1 .................................................................................................................. 11.1. ...................................................................................... 11.2. .......................................................................................................... 21.3. .......................................................................................................... 3 2 .......................................................................................... 52.1. .................................................................................. 52.1.1. ........................................................................................... 52.1.2. ............................................................................... 92.1.3.P2P .............................................................................. 122.2. ............................................................................................ 182.2.1. ......................................................................................... 182.2.2. ......................................................................................... 182.2.3. ......................................................................................... 292.2.4. ............................................................................. 352.3. ................................................................................ 36 3 ................................................................................................ 383.1. .................................................................................... 383.1.1. ......................................................................................................... 383.1.2. ................................................................................. 383.1.3. ................................................................. 413.1.4. ............................................................................. 423.2. ............................................................................................ 443.2.1. ......................................................................................................... 443.2.2. ................................................................. 443.2.3. ......................................................... 513.3. ................................................................................ 553.3.1. ......................................................................................................... 553.3.2. ..................................................................................... 553.3.3. ..................................................................... 57

IV3.4.ROA ............................................................................................... 643.4.1. ......................................................................................................... 643.4.2. ..................................................................... 643.4.3.ROA ........................................................................................ 663.4.4. REST ......................................................... 70 4 FFS ........................................................ 774.1.FFS ........................................................................................ 774.2. ........................................................................................................ 784.2.1. ................................................................................................. 784.2.2. ................................................................................................. 794.3. ........................................................................................................ 804.3.1. ......................................................................................... 804.3.2. ......................................................................................... 814.3.3. ................................................................................. 814.3.4. ..................................................................... 834.3.5.AC-RU .................................................................................... 854.3.6. ............................................................................. 864.3.7. ..................................................................... 874.4. ........................................................................................................ 874.5. ........................................................................................ 884.5.1. ..................................................................... 884.5.2. ..................................................................... 894.5.3. ............................................................................. 90 5 ................................................................................................ 925.1. ............................................................................................ 925.1.1. ................................................................................................. 925.1.2. ................................................................................. 935.1.3. ................................................................................. 945.1.4. ................................................................................. 955.2. FFS ................................................................................... 97 6 .................................................................................................... 99 .............................................................................................................................. ... 102 ................................................................................................................... 103 ........................................................................................................................... 10 4 1 ........................................................................................................... 109 2 ....................................................................................................... 111

V 1-1 ................................................................................... 1 2-1 DAS ........................................................................................................... 5 2-2 NAS ........................................................................................................... 6 2-3 SAN ........................................................................................................... 7 2-4 iSCSI IP-SAN ..................................................................................... 8 2-5 FCIP IP-SAN ...................................................................................... 9 2-6 NFS ................................................................................................. 10 2-7 Coda .................................................... 11 2-8 CFS .............................................................................................. 13 2-9 CFS ...................................................................................... 13 2-10 OceanStore .................................................................... 15 2-11 OceanStore BGUID ...................................................................... 16 2-12 OceanStore .................................................................................... 16 2-13 OceanStore .................................................................................... 17 2-14 GFS ............................................................................................... 19 2-15 ............................................................................................... 23 2-16 S3 ........................................................................... 24 2-17 Windows Azure ............................................................................. 24 2-18 Blob ................................................................................................... 25 2-19 HDFS ..................................................................................... 28 2-20 Hadoop .................................................................................. 29 2-21 ............................................................................... 30 2-22 ADFS ..................................................................................... 31 2-23 ADFS ................................................................................. 32 2-24 ADFS ..................................................................................... 33 2-25 TFS1.0 ....................................................................................... 34 2-26 ............................................................................... 35 3-1 SNIA ............................................................................... 38 3-2 ................................................................................. 41 3-3 ................................................................................. 42 3-4 Fuse ............................................................................................. 43 3-5 ............................................................................. 45 3-6 ......................................................................................................... 47 3-7 ......................................................................................................... 47

VI 3-8 ......................................................................................................... 49 3-9 m n ................................................................................ 50 3-10 m n .............................................................................. 50 3-12 / .................................................. 52 3-11 ............................................................ 52 3-13 x q ........... 53 3-14 X N Q(X,N) ....................... 54 3-15 X NBest ................................................................................... 55 3-16 X QBest Q(N=1) ................................................................. 55 3-17 ............................................................................................... 57 3-18 ........................................................................... 58 3-19 ............................................................................................... 58 3-20 ........................................................................................... 59 3-21 ............................................................................................... 61 3-22 ................................................................................................... 61 3-23 ................................................................................................... 62 3-24 ................................................................................... 64 3-25 REST ................................................................. 71 3-26 ................................................................................... 71 3-27 ................................................................................... 72 3-28 ............................................................................... 74 3-29 ................................................................................... 75 4-1 FFS .................................................................................. 78 4-2 ................................................................................................. 79 4-3 ................................................................. 81 4-4 FTP ..................................................................... 83 4-5 50 ................................................................. 83 4-6 .......................................................... 84 4-7 .................................................................. 84 4-8 AC-RU ..................................................................................... 85 4-9 ..................................................................................... 86 4-10 ........................................................................... 87 4-11 ....................................................................................................... 88 4-12 ........................................................................... 89 4-13 ................................................................................................... 89 4-14 ................................................................................................... 90

VII 4-15 ............................................................................................... 90 4-16 ............................................................................................... 91 5-1 ............................................................................................. 92 5-2 2 / ................................................................ 93 5-3 16 / .............................................................. 93 5-4 9 / ................................................................ 93 5-5 / .................................................................................... 93 5-6 ............................................. 94 5-7 ............................................. 95 5-8 ................................................. 96 5-9 ..................................................... 96 5-10 ............................................................... 97 6-1 ......................................................................................... 99 6-2 ....................................................................................... 100 6-3 ....................................................................................... 101

VIII 2-1 CFS .......................................................................................... 12 2-2 ............................................................................................. 22 2-3 S3 ..................................................................................................... 23 2-4 ............................................................. 29 2-5 ..................................................................................... 35 3-1 GF(23) .............................................................................................. 48 3-2 - -Mbps ..................................... 50 3-3 X NBest QBest .................................................................................. 54 3-4 ................................................................................................. 63 3-5 / ............................................................ 65 4-1 ..................................................................................... 79 4-2 ......................................................................... 86 4-3 ......................................................................................................... 88 5-1 ................................................................................................. 92 5-2 FTP .............................................................................. 97

1 1 1.1. Jim Gray 18 [1] 1-1 IDC 2006 2010 10 161EB 1580EB 100 I/O 30 50 161 280 540 988 1580 2529 0 500 1000 1500 2000 25003000 20042005200620072008200920102011E (E Byte) 1-1 1TB DAS/NAS/SAN [2] TB PB PAL 25MB/s MPEG-2 0.5MB/s [3]

2 (single point of failure) PB 1.2.

3 PB Scalability Universality PC High-availability High-performance I/O Self-management Convenience 1.3. 6 1 2 3 5 Fuse Dokan RS ROA REST 4 FFS 3 FFS AC-RU FFS

4 5 FFS I/O FFS 6 FFS

5 2 Web Servers Mail Servers Video On Demand, VOD Digital Library Geographic Information System, GIS Database Servers Internet Data Center, IDC P2P 2.1. P2P 2.1.1. DAS, Direct-Attached Storage NAS, Network Attached Storage SAN, Storage Area Network IP IP-SAN 2.1.1.1. DAS DAS DAS RAID DAS DAS 2-1 IDE SATA SCSI 2-1 DAS DAS

6 DAS DAS I/O DAS DAS DAS 2.1.1.2. NAS NAS 2-2 NAS NAS CPU NAS NFS CIFS SCSI FC 2-2 NAS NAS IP NFS Network File System CIFS Common Internet File System NAS I/O NAS NAS I/O Block I/O NAS NAS [4] 1

7 2 3 NAS RAID SAN NAS RAID SAN 4 NAS [4] 1 NAS File I/O I/O File I/O TCP/IP File I/O NAS TCP/IP NAS NAS 2 NAS NAS 3 NAS 2.1.1.3. SAN SAN Network-Centered [5] SNIA SAN Fiber Channel, FC SAN FC IP SAN Block I/O SAN 2-3 2-3 SAN SAN [6][7] 1 2 10Gbps 40Gbps

8 3 SAN 1500 4 5 6 LAN Free Backup SAN SAN [6][7] 1 SAN 2 10km 3 SAN SAN 4 SAN Fiber Channel 5 SAN 2.1.1.4. IP IP-SAN SAN Block I/O NAS File I/O IP SAN NAS IP Block I/O IP-SAN iSCSI FCIP Fiber Channel over IP IP [8] iSCSI SCSI IP iSCSI HBA 2-4 IP IP SCSI I/O iSCSI SCSI I/O FC iSCSI I/O 2-4 iSCSI IP-SAN

9FCIP TCP/IP FCIP SAN SAN 2-5 2-5 FCIP IP-SAN IP-SAN [9] 1 IP-SAN IP 2 SCSI TCP/IP 3 SAN LAN Internet 4 IP-SAN SAN 2.1.2. 20 90 NFS AFS Coda NFS 2.1.2.1. NFS NFS Network File System Sun Microsystem 1985 [10] UNIX Linux FreeBSD Windows VMS NFS NFS C/S NFS NFS

10 NFS 2-6 mount 2-6 NFS NFS Stateless NFS NFS NFS I/O NFS NFS Cascading Mount NFS NFS NFS 2.1.2.2. AFS AFS Andrew File System IBM Camegie Mellon University [11] AFS 3.0 IBM AFS AFS AFS AFS AFS AFS /afs AFS Cell Cell Cell AFS AFS AFS AFS AFS

11 AFS AFS / 200:1 I/O AFS Cell 2.1.2.3. Coda Coda 1987 AFS Coda AFS [12] Coda Disconnection [13] 2-7 Coda 2-7 Coda Volume VSG Volume Storage Group Coda Venus Venus AVSG Accessible Volume Storage Group AVSG VSG Venus AVSG Venus Venus AVSG CPU Coda Read-one, Write-all AVSG Coda Coda

12 VSG AFS VSG Coda Coda sticky sticky 2.1.3. P2P [14] Internet Internet C/S P2P Peer-to-Peer [15] Peer P2P Peer P2P Chord[16] Tapstry[17] P2P P2P P2P P2P CFS OceanStore Past 2.1.3.1. CFS CFS[18] Chord Chord CFS C/S block CFS CFS CFS 1 FS File System 2 DHash 3 Chord 2-1 2-1 CFS FS DHash

13DHash Chord Chord CFS FS DHash Chord 2-8 Remote Procedure Call, RPC 2-8 CFS CFS DHash CFS CFS pre-fetching CFS DHash 2-9 2-9 CFS CFS CFS root-block CFS DHash CFS CFS

14DHash k k k DHash k Chord k DHash Chord r k k 1 ID 2 k CFS DHash CFS (log) ON N Chord DHash LRU least-recently-used CFS CFS Chord 2.1.3.2. OceanStore OceanStore[19] Tapstry OceanStore OceanStore OceanStore OceanStore OceanStore Byzantine Fault Tolerant Commit Protocol [20] OceanStore API OceanStore

15 erasure code OceanStore OceanStore introspection OceanStore regional outage DoS OceanStore OceanStore OceanStore 2-10 [21] VGUIDi Version GUID i i backpointer AGUID Active GUID AGUID OceanStore BGUID Block GUID B root block M BGUID VGUID indirect blocks data block BGUID OceanStore ID BGUID copy on write 2-10 OceanStore 2-11 OceanStore BGUID 4 H1 H2 H3 H4 H1 H2 H12 H3 H4 H34 H12 H34 H14 H14

16Hd GUID BGUID log N N 1 H2 H34 Hd 1 H1 H2 H12 H12 H34 H14 H14 Hd BGUID 2-11 OceanStore BGUID 2-12 OceanStore [19] OceanStore 1 observation 2 optimizatin 3 computation 2-12 OceanStore 2-13 [19] Event Stream Event Handlers DB Periodic In-Depth Analysis Issue Local Optimizations Commands Forward Aggregates for Further Processing

17 2-13 OceanStore P2P OceanStore Cluster recognition 2.1.3.3. PAST PAST[22][23] P2P Pastry[24] Internet Pastry PAST ID ID 56Kbps 10Gbps 1KB GB PAST PAST smartcard / smartcard issuer PAST PAST Pastry Pastry Pastry [24] Pastry PAST k

18 PAST PAST GD-S GreedyDual-Size [22] Web GD-S d weight H(d) d H(d) c(d)/s(d) c(d) d s(d) H(d) v v H(d) H(v) GD-S 1 GD-S 2.2. 2.2.1. Cloud Computing [25] Cloud Storage 2.2.2. IT

192.2.2.1. Google Google [26] Google File System GFS Google Chubby MapReduce Bigtable GFS GFS Google GFS GFS GFS GFS GFS 2-14[26] GFS Client Master Chunk Server Client GFS Master GFS Chunk Server Chunk Server Chunk Server GFS GFS 64MB Index 2-14 GFS GFS Master Chunk Server Chunk Server GFS Master Client Master Client Chunk Server Client Chunk Server I/O

20 GFS Google [26] 1 GFS Master Chunk Master Chunk Server Client Master Chunk Server Chunk Server Master Chunk Server Chunk Server Chunk Server Master Chunk Server GFS Master Master 2 GFS Chunk Server GFS Chunk Server GFS Chunk Server Master GFS GFS Client Master Master Master 3 Linux Ext3 Lustre[27] POSIX GFS POSIX

21 POSIX Master Chunk Server GFS GFS GFS GFS Master Chunk Server 1 Master Master GFS Name Space Chunk Chunk Chunk Master GFS Chunk Server Chunk Server Master Master Master GFS Master Master Master 2 Chunk Server GFS Chunk Server Chunk Chunk Server Chunk Master Chunk Server GFS Chunk Chunk 64MB Google 64MB Chunk Server Chunk Chunk Block 64KB Block 32bit Chunk Chunk Server

22 Client Chunk Server 2.2.2.2. Amazon S3 Simple Storage Services [28] Amazon S3 S3 S3 Dynamo[29] S3 Object Key Bucket S3 S3 System Metadata User Metadata 2048B S3 2-2[30] 2-2 last-modified ETag MD5 Content-Type MIME / Content-Length - Name-Value S3 5GB UTF-8 1024B [31] 100 S3 S3 IP

23 S3 PC S3 S3 2-15 2-15 S3 Get Put List Delete Head 2-3[30] 2-3 S3 Get Put List Delete Head S3 [31] S3 S3 S3 S3 S3 S3 S3 S3 S3 2-16

24 2-16 S3 2.2.2.3. Windows Azure Blob Windows Azure [32][33] Windows Azure Blob[34] Table[35] Queue[36] Blob Table SQL Queue MSMQ Windows Azure Windows Azure 2-17[32] 2-17 Windows Azure Windows Azure Blob

25 Blob Blob Blob Blob URI URI [34] http://.blob.core.windows.net// Blob Storage Account URI Container Blob Blob 50GB Blob Block Blob Blob 2-18 2-18 Blob Blob HTTP REST PUT/GET/DELECT [34] Blob 1 PUT Blob Blob Blob 2 GET Blob Blob HTTP Range GET Blob 3 DELETE Blob Blob PUT 64MB Blob 64MB Blob Windows Azure Blob Block Blob Block 4GB 1000 Block 4MB Block ID Block 1 Block 2 2-18 Block Blob Blob Block ID Blob Block ID Block 4MB Block Blob Block Blob Block Block Blob Block Block Windows Azure Storage Block Blob Block Blob

26 2-18 MOVIE URI http://MyAcount.blob.core.windows.net/Mov /MOVIE.RMVB Windows Azure Blob Block PUT Block PUT Block List GET Block List PUT Block Block REST PUT Block [25] REST PUT Block PUT http://MyAcount.blob.core .windows.net/Mov/MOVIE.RMVB ?comp=block &blockid=BlockId_0001 &timeout=60 HTTP/1.1 Content-Length: 4194304 Content-MD5: HUXZLQLMuI/KZ5KDcJPcOA== Authorization: SharedKey MyAcount: F5a+d UDvef+PfMb4T8Rc2jHcwfK 58KecSZY+l2naIao= x-ms-date: Mon, 27 Oct 2008 17:00:25 GMT Block Data Contents PUT HTTP "?comp=block" PUT Block blockid=BlockId_0001 blockid Content-MD5 Block Content-Length Authorization Block PUT Block Block Block BlockID Block Blob Block PUT http://Myaccount.blob.core .windows.net/Mov/MOVIE.RMVB ?comp=blocklist &timeout=120 HTTP/1.1 Content-Length: 161221 Authorization: SharedKey sally: QrmowAF72IsFEs0GaNCtRU143JpkflIgRTcOdKZaYxw= x-ms-date: Mon, 27 Oct 2008 17:00:25 GMT <?xml version="1.0" encoding="utf-8"?> BlockId_0001BlockId_0002 ?comp=blocklist Block PUT HTTP blocklist Block MyAccount Mov Blob Blob MOVIE.RMVB Blob Blob GET http://sally.blob.core .windows.net/Mov/MOVIE.RMVB HTTP/1.1 Authorization: SharedKey sally: RGllHMt zKMi4y/nedSk5Vn74IU6/fRMwiPsL+uYSDjY= X-ms-date: Mon, 27 Oct 2008 17:00:25 GMT

27 Blob GET HTTP Range Range: bytes=1024000-2048000 Block Block blocklist Block Blob Blob Block Blob ID Block Block Blocklist 2.2.2.4. HDFS Hadoop HDFS Hadoop Distributed File System [37] Google GFS TB PB MapReduce HDFS HDFS NameNode DataNode NameNode DataNode NameNode I/O DataNode NameNode Namespace DataNode HDFS HDFS 64MB HDFS 64MB Block DataNode NameNode DataNode DataNode NameNode 2-19[25] HDFS NameNode DataNode DataNode NameNode

28 2-19 HDFS HDFS NameNode DataNode NameNode DataNode DataNode NameNode HDFS 4GB NameNode NameNode Checkpoint HDFS Secondary NameNode NameNode NameNode Secondary NameNode NameNode NameNode NameNode Hadoop Hadoop [37] Hadoop n1 d1 r1 /d1/r1/n1 /d1/r1/n1 /d1/r1/n1 =0 /d1/r1/n1 /d1/r1/n2 =2

29 /d1/r1/n1 /d1/r2/n3 =4 /d1/r1/n1 /d2/r3/n4 =6 2-20 2-20 Hadoop 2.2.3. IT Springboard Research 2010 8 China Cloud Storage Services Report 5 103% 2009 605 2014 2.1 IT 2-4 DBank 115 360 360 IM QQ 2-4

30 ADFS TFS 2.2.3.1. B2B 2009 IBM Google ADFS Alibaba Distributed File System [38] ADFS 100KB GFS HDFS Master 100MB ADFS GFS GFS 1MB 2GB ADFS GFS 1 ADFS GFS offset { ID} 0 N ID 1 16KB 20 2-21 2-21 ID+ +{ ID}+ ID+ +

31 200MB 64MB 4 4 1001 1002 1003 1004 1001 0 1002 1003 1004 0 515 Master 100KB 64MB Master 100KB ID 1102 100 100*16=1600KB 3996 3996 7 100/16+1=7 20 1002 3996 7 2 Master Master ADFS Master Master Master 2-22 2-22 ADFS 2-22 ADFS Master Master Master Master Master

32 Master 3 2-23 ADFS ADFS API /ADFS/tmp/test.txt ADFS API Create Master /ADFS/tmp/test.txt Master ADFS Master 2-23 ADFS 2-24 ADFS ADFS API /ADFS/tmp.test.txt API Master Master API API API API

33 2-24 ADFS 2.2.3.2. [39] Taobao File System, TFS 90% 10%. TFS 1.8PB 990TB 286 17.45KB 8KB 61% 11%. 2007 NetApp 3 NetApp 2006 NetApp 2006 2007 6 TFS1.0 200 PC Server 140TB 50TB 3MBPS 2-25 TFS1.0 TFS TFS Name Server Data Server Name Server Data Server Linux

34 Block Block 64MB Block Data Server RAID5 2-25 TFS1.0 TFS 1 TFS ID 2009 6 TFS1.3 200 PC 440 PC 1.8PB 995TB 15MBPS 2-26 TFS 200 Apatch 90%

35 2-26 TFS TFS 2.2.4. 6 2-5 2-5 GFS HDFS Amazon S3 Microsoft BlobADFS TFS Blob REST Blob

36 Blob Blob Block MapReduce PC 2.3. [40]

37[41] DAS NAS SAN IP-SAN NFS AFS Coda P2P CFS OceanStore PAST SAN SAN

38 3 3.1. 3.1.1. 70 RAID / 3.1.2. SNIA 3-1 3-1 SNIA

39 3.1.2.1. 3-1 1 [42] LUN LUN LUN LUN 2 [42] 3.1.2.2. 1 [43] 2 [43]

40 3 [44] SAN I/O 3.1.2.3. 1 [45] 2 [45] I/O I/O I/O /

413.1.3. 3-2 3-2 3-2 /

423.1.4. 3-2 3-3 Fuse[46] Linux Dokan[47] Windows 3-3 3.1.4.1. Fuse Fuse (Filesystem in Userspace) Dokan Fuse Windows Fuse Unix Linux 2.6.9 Fuse Fuse Fuse (fuse.ko) (libfuse.*) 3-4

43 3-4 Fuse 3-4 hello fuse /tmp/fuse ls l /tmp/fuse VFS Fuse hello Fuse Linux VFS Ext2 Ext3 Fuse VFS Fuse Fuse VFS Fuse Fuse API Ext2 Ext3 Fuse FTP Https Fuse 3.1.4.2. Dokan Dokan Fuse Windows Dokan DLL dokan.dll dokan.sys Dokan CreateFile ReadFile WriteFile Dokan dokan.dll

44 I/O Windows OpenDirectory Dokan OpenDirectory OpenDirectory Windows Dokan 3.2. 3.2.1. [48] [49] 3.2.2. IDA Information Disperal Algorithm Weatherspoon[50] Weatherspoon IO IO RS 3.2.2.1. L F n Fi Fi m F

45 n Fi 3-5 k n k k 3-5 [51] [n,k,d] d d-1 n k 1 1[52] G (n,k) C C G k x c k k G C x M x 3.2.2.2. RS 1960 I.S.Reed G.Solomon Reed-Solomon RS k RS Vandermond Code [53] Cauchy Code [54] 1 p r G k

46 k G k 3-1 000 011 111 011 111 011 n n kkk nxxx xxx G xxx (3-1) G y x y 2 2[55] G G G 3-2. 1 112 1 00000000 0110111 11111111 0110111 11111111 0110111 0,[|] 100 000 001nkkn nkkn kkkkkkkk nkkn kGVVV gggggggg gggggggg gggggggg g g 0,1 1,1,1 1,1,1 n kn kkkng g gg (3-2) 3 (1) (2) 3-3

47 111111 111111 111111111 100 111 000 111 001 x yxyxy x yxyxy G x yxyxy (3-3) 1[54] C C 1 3-6 3-7 D1 D2 D3 D4 D5 * 1 0 0 0 0 1 5 0 1 0 0 0 5 1 0 0 1 0 0 2 3 0 0 0 1 0 7 4 0 0 0 0 1 4 7 = D1 D2 D3 D4 D5 C1 C2 3-6 D1 D2 D3 D4 D5 * 1 0 0 0 1 0 1 0 0 5 0 0 1 0 2 0 0 0 0 7 0 0 0 1 4 = D1 D2 D3 D5 C1 D4 C2-1 3-7 3.2.2.3. m [56] 0 m 0 0 1 254 (00000000) (00000001)

48 3-1 3-1 3 256 8 3-1 GF(23) mod(a3+a+1) 0 0 (000) a 0 1 (001) a 1 a (010) a 2 a 2 (100) a 3 a+1 (011) a 4 a 2 +a (110) a5 a 2 +a+1 (111) a 6 a 2 +1 (101) a7 1 (001) a 8 a (010) 3-1 i i i 1Byte ; ;

49 5 3.2.2.4. L bytes F n Fi bytes 3-8 3-8 3-8(a) 2 4 3 RS RS m m n C S G m 5 3 3 r L g

50 3-8(b) 2 4 L 2L 3 RS RS m m [57] 3 L g 3-8 n Fi n / CPU Intel 2.33GHz 2GB 3-2 3-2 - -Mbps (MB) m=4 n=6 r=0.5 m=5 n=10 r=1 m=8 n=12 r=0.5 1 0.453 17.6600.539 14.8420.71111.2510.53714.8970.547 14.625 0.687 11.644 5 2.196 18.2142.570 15.5643.41111.7262.79514.3112.595 15.414 3.590 11.14210 4.383 18.2525.128 15.6006.77211.8135.34714.9615.219 15.328 7.001 11.42650 21.725 18.41124.933 16.04233.71311.86427.43514.57925.958 15.409 34.584 11.566100 43.534 18.37649.677 16.10468.19811.73054.25814.74451.318 15.589 68.258 11.720 3-2 m=4 n=6 / 18/15(Mbps) m=5 n=10 / 11/14(Mbps) m=8 n=12 / 15/11(Mbps) 3-2 m n / 3-9 3-10 m n m n m n 1 4 7 10 13 16 19 22 25 28 30 1 2 3 4 5 6 7 8 9 10 0 10 20 30 40 50 n m(Mbps) 3-9 m n 1 4 7 10 13 16 19 22 25 28 30 1 2 3 4 5 6 7 8 9 10 15 17 19 21 23 25 n m(Mbps) 3-10 m n RS / CPU

51 / CPU Raptor[58] Raptor LDPC[59] / 100% 3.2.3. / / / 3.2.3.1. RAS RAS Reliability Availability Serviceability [60] [61] MTBF mean time between failure MTTR mean time to repair MTBF MTTR Availability (3-4) (3-4) / / 3-11 / MTBF

52 (3-5) 3.2.3.2. x (GB) v (GB/s) / (3-5) / (3-6) 3-12 MTBF=21(h) MTTR=3(h) v=0.5MB/s x / Px 0 10 20 30 40 50 0 0.5 1 x(GB)Px v * MTBF MTBF/(MTBF+MTTR) 3-12 / 3-12 / Availability / / n MTBF MTTR t 3-11

53 (3-7) (3-6) (3-7) (3-8) (3-8) n 3-13 MTBF=21(h) MTTR=3( ) v=0.5MB/s n 1 3 6 9 4 x q 0 10 20 30 40 50 0 0.5 1 x/GBq 0 10 20 30 40 50 0 0.5 1 x/GBq (a) n=1 (b) n=3 0 10 20 30 40 50 0 0.5 1 x/GBq 0 10 20 30 40 50 0 0.5 1 x/GBq (c) n=6 (d) n=9 3-13 x q 3-13 q 3.2.3.3. X N x=X/N n

54 (3-9) 3-14 MTBF=21(h) MTTR=3(h) v=0.5MB/s n=3 X N 0 10 20 30 40 50 1 10 20 30 40 50 60 70 80 90 100 0 0.2 0.4 0.6 0.8 1 N X/GB Q 3-14 X N Q(X,N) 3-14 N NBest QBest X NBest QBest (3-9) N 0 N (3-10) MTBF=21(h) MTTR=3(h) v=0.5MB/s n=3 Matlab X (3-10) NBest X NBest (3-9) QBest 3-3 X NBest QBest 3-3 X NBest QBest X /GB 1 2 4 8 16 32 50 NBest 1 1 1 3 5 10 16 QBest 0.97910.9687 0.94010.88570.78530.62630.4701 QN=1 0.97910.9687 0.94010.85090.57090.04020 3-3 N=1 Q X

55 3-15 X NBest 3-16 X QBest Q(N=1) 0 10 20 30 40 50 0 5 10 15 20 X/GB NBest 3-15 X NBest 0 10 20 30 40 50 0 0.5 1 X/GB Q Q(N=1) QBest 3-16 X QBest Q(N=1) 3-15 3-16 3.3. 3.3.1. 3.3.2. IBM IBM Paul Horn 2001

56[63] [64] QoS [64] IT IT IT IT IT IT IT IBM IT 4 [64] 3-17

57 3-17 3.3.3. 3.3.3.1. ID ID 3.3.3.2. 3-18

58 3-18 CPU 3-19 3-19

59 3-20 register heartbeat-reactor state-sensor filter data-wrapper 3-20 [ ID, Time, StateType, Value ] ID Time StateType CPU Value ID StateType

60[ NodeType, StateType, CurrentState, Operation, NextState ] NodeType StateType CurrentState Operation NextState CS3721 90% [CS3721, 13:25:06, DISKUR, 0.9] CS3721 DISKUR (1) [ChunkServer, DISKUR, Heavy, PreventNewStore, Heavy] (2) [ChunkServer, DISKUR, Heavy, DataTransfer, Normal] (1) Operation =PreventNewStore (2) Operation =DataTransfer NextState =Normal Heavy Normal [ ID, Time, Operation, Argument1, Argument2, ] ID Time Operation Argument 0 [CS3721, 13:25:06, DISKUR, 0.8] { [ChunkServer, DISKUR, Heavy, PreventNewStore, Heavy] [ChunkServer, DISKUR, Heavy, DataTransfer, Normal] } Heavy [ChunkServer, DISKUR, Heavy, PreventNewStore, Heavy] [CS3721, 13:25:16, PreventNewStore]

61 3-21 3-21 3-22 register filter command-parser transactor behavior-library 3-22

62 3-23 3-23 1 (,,)i M IDRA i M I D R A 2 1(,,)A in AP Axx E iA 1,,n x x AP AE 3 0(,,)ikkSssr iSS S

63 iS 0s ks kr 3.3.3.3. (,) NST 12(,,,)nSsss 12(,,,)nTttt 3-4 3-24 t0s1 t1(i)is2(i) it2(i)is3(i)it3(i) s2(i)s4(i) it4(i)is5(i)it1(i) 3-4 s0 s1 s2(i) i s3(i) s4(i) i s5(i) i t0 t1(i) t2(i) i t3(i) t4(i) i

64 3-24 3.4. ROA 3.4.1. 3.4.2. POA OOA SOA ROA POAOOASOAROA

65SOA RPC POAROA OOA3-5 3-5 / / POA SOA / OOA ROA 1 2 3 4 5 Windows Linux 6

66 Roy Fielding[65] ROAREST ROAROA ROA 3.4.3. ROA 3.4.3.1. ROA ROA[65] 1 2 URIURIURIURI URI URIURIURI user1file1.txtURI/user1/root/file1.txt 3 URIURI URIuser1 file1.txtURI/user1/root/file1.txt/metadata 0~512URI

67/user1/root/file1.tx t/chunks/2/0-512. 4 3.4.3.2. ROA ROA [65] 1 URIURI URI 2 URI URI 3 ROA Amazon S3ROA S3URIS3bucket bucketURIbucket bucketsURI 4 GETPUTPOSTDELETE PUTURIPOST PUTURI URIPOST

68URIURI 5 0 4000 40 PUTDELETEPUT 6 1 414GET GET GETGET PUT PUTPUTPUT 3.4.3.3. REST ROAROA ROA REST Representational State Transfer RESTRoy Fielding[65] Roy Fielding RESTWeb HTMLXML RESTWeb[66] 1 resource 2resource identifier 3generic connector interface 4

69 5stateless 2WebURL3 WebHTTPGETPOSTPUTDELETE WebSOAPWeb REST- [65] 1- - - 2 3 3 4 REST REST

70Web 5 Internet 6 RESTapplet REST REST 3.4.4. REST RESTRESTWebRESTROAWebHTTPURI REST 3.4.4.1. REST 3-25REST1 - REST5 REST 3 GETPUTPOSTDELETE REST4

71URI 3-25 REST 3.4.4.2. 3-26 3-26 1 URI /master/userlist PUTDELETEGETPOST PUT

72 PUT /master/userlist PUT DELETE 2 URI/master/userlist/username GET GET GET /master/userlist/username password=xxxxxxx ID 3.4.4.3. 3-27 3-27

731 URI /supervisor/metadataserverlist POST POST POST /supervisor/metadataserverlist/ 1 2 URI /supervisor/chunkserverlist POST POST POST /supervisor/chunkserverlist/ 3 URI/supervisor/metaserver GET GET /supervisor/metaserver 4 URI/supervisor/chunkserver GET GET /supervisor/chunkserver 1

743.4.4.4. 3-28 3-28 1 URI/metadataserver/state GET GET GET /metadataserver/state/ 2 URI/metadataserver/username/ GETPUTDELETE POST GET GET /metadataserver/username/folderX//folderY/ < name=xxx attribute=xxx createtime=xxx modifytime=xxx />

75ID ID PUT PUT /metadataserver/username/folderX//folderY/ DELETE DELETE /metadataserver/username/folderX//folderY/ 3.4.4.5. 3-29 3-29 xxxxxx 1 URI/chunkserver/state GET GET

76 GET /chunkserver/state/ 2xxx xxx URI/chunkserver/store/xxx GETPUTDELETEPOST GET GET /chunkserver/store/xxx PUT PUT /chunkserver/store/ DELETE DELETE /chunkserver/store/xxx 3xxx xxx URI/chunkserver/store/xxx/aaa[-bbb] GETPUTPOSTDELETE GETxxxaaabbb GET /chunkserver/store/xxx/aaa-bbb PUTxxxaaa PUT /chunkserver/store/xxx/aaa xxxx

77 4 FFS 4.1. FFS DAS/NAS/SAN Formicary File SystemFFS FFS FTPWeb FFS FFS FFS FFS FFSFFS FFS

78 4.2. 4.2.1. FFS4-1FFS FFS / 4-1FTP Web 4-1 FFS FFS4-2 CPU /

79 4-2 4.2.2. FFS94-19 4-1 1 2 3 Read Write WMI 4 5 Read Bug IOPS 10 40 6 CPU 7 8 9 9FFS4.15FFS

80FFS FFS FFS 4.3. PCFFS AC-RUAdjust-Controlled Recently-Used 4.3.1. 4-3 state state 2 state1state0

81 4-3 4.3.2. FFS CPU/ 4-3 FFS 4.3.3. FFS FFS

82 // (4-1) ()iiii ilaudn (4-1) liiai uidini uidinikbps1 LThresholdUThreshold 1[()]N ii i i L udn (4-2) Nuidini(4-1) 2 1()N i iaa U N (4-3) ai(4-1)N(4-2) a LThresholdLMax 0 LLThreshold=LMax*20%. UThreshold Round-RobinN NURRUThreshold=URR*80% FTP

83FTP251692612424.75TB4-4 0 34.29GB24.3 e5~e20(e2.718)50 250GB4-5 URR=6.88UThreshold=5.504 0 5 10 15 20 25 0 1 2 3 4 x 104 4-4 FTP 0 10 20 30 40 50 0 0.5 1 1.5 2 x 1011 4-5 50 4.3.4. PC FFSFFS FFSRead-one Write-all[62]

84 FFS 4-6 4-6 / GUIDGUID IPstateCPU/// 4-7 // / 4-7

85 4.3.5. AC-RU AC-RUAdjust-Controlled Recently-Used 4-8AC-RUAC-RU AC-RU 4-8 AC-RU AC-RU8AC-RUN N/2FIFO(First In First Out)[67]N/2LRU(Least Recently Used)[67] FIFO LRU N/2 N/2AC-RULRU

86LRU 4.3.6. FFS FFS4-9FFS WindowsLinux 4-9 Linux Fuse WindowsDokanFuseLinux FuseDokanWindowsFuseFuseDokan 3.14-2Dokan 4-2 Cleanup CloseFile CreateDirectory CreateFile DeleteDirectory DeleteFile FlushFileBuffers FindFiles GetFileInformation LockFile MoveFile

87 OpenDirectory ReadFile WriteFile / SetEndOfFile SetAllocationSize SetFileAttributes SetFileTime UnlockFile Unmount GetDiskFreeSpace 4.3.7. FFS 4-10 4-10 Active Standby XML XML 4.4. FFSUDP UDT (UDP-Based Data Transfer[68][69] 4-11FFS

88 4-11 UInt16FFSFFS ByteFFS 4-3FFS19 2 4-3 DEV_JOIN HEARTBEAT GET_CHUNKSV DISKSPACE FOLD_EXIST FILE_EXIST UPDATE_SIZE FOLDINFO FILEINFO CREAT_FOLD CREAT_FILE_M CREAT_FILE_S DEL_FOLD DEL_FILE_M DEL_FILE_S MOV_FOLD MOV_FILE READ WRITE 4.5. 4.5.1. //4-12

89 4-12 4.5.2. FFS/4-13 UDT 4-13 4-14UDT

90 4-14 4.5.3. 4-15FFS 4-15 4-16

91 4-16

92 5 5.1. 5.1.1. FFS 1 1616100MbpsHUAWEI Quidway S3000 1Gbps 10 15-1 5-1 IntelIOMeter 5-1 CPU Core2 2.6GHz4GB250GBWindows Server 2003 64 1 Core2 2.6GHz2GB2TB Windows XP 32 16 Core2 2.6GHz2GB250GBWindows XP 32 16 5-1

935.1.2. 5-2~5-3IOMeter 291616100Mbps 12.5MB/s1Gbps125MB/s 225MB/s9 112.5MB/s16125MB/s1Gbps 0 2 4 6 8 10 12 14 16 0 20 40 60 80 100 120 /MB/s 5-2 2 / 0 2 4 6 8 10 12 14 16 0 20 40 60 80 100 120 /MB/s 5-3 16 / 0 2 4 6 8 10 12 14 16 0 20 40 60 80 100 120 /MB/s 5-4 9 / 2 9 16 0 20 40 60 80 100 120 /MB/s Write Read 5-5 / 9.4MB/s 75%6.1MB/s 48% 5-529162 21.9MB/s11.7MB/s9

9488.9MB/s25.6MB/s16105MB/s49.4MB/s 5.1.3. FFS4 2.5MB 1 5-6 % AB60CAB120D 20 40 60 80 100 120 140 160 180 200 0 50 100 20 40 60 80 100 120 140 160 180 200 0 50 100 20 40 60 80 100 120 140 160 180 200 0 50 100 20 40 60 80 100 120 140 160 180 200 0 50 100 C D B A 5-6 5-7 ABCD260 A3A 300 320BCD

95 200 220 240 260 280 300 320 340 360 380 400 0 50 100 200 220 240 260 280 300 320 340 360 380 400 0 50 100 200 220 240 260 280 300 320 340 360 380 400 0 50 100 200 220 240 260 280 300 320 340 360 380 400 0 50 100 A B C D 5-7 5-65-7FFS5-7 5.1.4. FFS 1616 18MB 1610560 1 5-8

96 0 500 1000 1500 2000 2500 3000 3500 0 5 10 15 20 0 500 1000 1500 2000 2500 3000 3500 0 15 30 45 60 75 85 5-8 5-9 0~600 LThreshold UThreshold=5.504600~900 LThreshold UThreshold UThreshold5-85-9 0 500 1000 1500 2000 2500 3000 3500 0 5 10 15 20 0 500 1000 1500 2000 2500 3000 3500 0 5 15 30 45 60 75 85 5-9

975.2. FFS FFSIDC FFS FTP5-10 5-10 FFSFtpServWindows Server 2003 CPUIntel Core 2 Dual2GBFTPFFS1CPUIntel Core 2 Quad3.21GB 5CPUIntel Core 2 Dual2GB1TB9 DNSFTP 3IPFTP FTP DNSFTP FTP FTPFTP 1GBFTPFTP1GB5-2FTP 5-2 FTP MB/S 25 123.16 29 106.77

985-2/123.16MB/S106.77MB/S FFS FTPFFS

99 6 6-1 6-1 6-2

100 6-2 FuseDokan RS ROAREST 6-3

101 6-3 FFSFFS FFS PC AC-RU FFSIO FFS FFSFFS FFS FFS

102 FFS FFS Linux Web

103 [1] Haijia Wu, Peng Liu, Weiwei Chen. The Op timization Theory of File Partition in Network Storage Environment. The 9th International Conference on Grid and Cloud Computing. Nanjing, Jiangsu, P. R. China. 2010.11. [2] , . RS. . 2010.12: Vol. 30. No. 12. [3] , , . . . 2010.10: Vol. 11. No. 4. [4] , , , , . FFSPB . . 2011.x: [5] , , , . . SaaS. 2011.8, [6] , , . MassCloud. . 2011.x: [7] , , . . . 2011.x: [8] , , . Web. . 2011.x: [9] , , . Web. . 2010.8: 30(8). [1] , , . . . . [1] PB [1] 2110 [2] 360 [3] [4] [5] PB

104 [1] Jim Gray. What Next? A Few Remaini ng Problems in In formation Technol ogy[EB/OL]. 2010.12. http: //research.microsoft.co m/~gray/Gray_Turing_FCR C.pdf. [2] W. Crutis Preston. Using SANs and NAS [M]. Sebastopol, CA: OReilly Media, 2002. [3] Calvin Zito. Closing Critical IT Ga ps and Driving Towards Converged Infrastructure with Storage Virt ualization[EB/OL]. 2009.11. http:// h18006.www1.hp.com/storage/4AA0-2062ENW. [4] . [D]. : , 2008: 4-5. [5] M. Farley. SAN()[M]. , , . : , 2002: 205~273. [6] , , , . iSCSi [J]. , 2004, 41(1):207~213. [7] , , . iSCSISAN[J]. , 2003, 40(5):246~251. [8] G. Somasundaram, Alok Shrivastava. [M]. , , . : , 2010: 171~173. [9] . [D]. : , 2004: 6. [10] Russel Sandlberg. The Sun Network File System: Design, Implementation and Experience[A]. Proceedings of the 1987 Summer Usenix Conference[C], Summer 1087, University of California Press, Pages 300~314. [11] Sidebotham. B. Volumes: The Andrew File System Data Structuring Primitive[R]. Technical Report CMU-ITC-053, Carnegie Melon University, 1986. [12] M. Satyanarayanan, J. J. Kistler, P. Kuma r. Coda: a Highly Available File System for a Distributed Workstation Environmen t[J]. IEEE Transactions on Computers, 1990. [13] J. J Kistler, M. Satyanarayanan. Disconnect ed operation in the coda filesystem[J]. ACM Transactions on Computer Sy stems, 10(1): 3~25, Feburay 1992. [14] D. G. Andersen, H. Balakrishnan, M. Kaashoek, et al. Resillent overlay networks[A]. In Proceedings of 18th ACM SOSP[C], Banff, Canada, October 2001. [15] , . [M]. : , 2007: 82~121.

105[16] Stoica, Robert Morris, David Karger, et al. Chord: A s calable peer-to-peer lookup service for internet applications[A]. In Proceedings of the ACM SIGCOMM 01 Conference[C], San Diego, Califonia, August 2001. [17] Ben Y. Zhao, John D. Kubiatowicz, Anthony D. Joseph. Tapestry: An Infrastructure for Fault-tolerant Wide -area Location and R outing[R], U. C. Berkeley Technical Report UCB//CSD-0101141, April 2001. [18] Frank Dabek, M. Frans Kaashoek, David Karger, et al. Wide-area cooperative storage with CFS[A]. In SOSP01[C], 2001: 202~215. [19] John Kubiatowicz, David Bindel, Yan Chen , et al. OceanStore: An Architecture for Global-Scale Persistent Stor age[A]. In: ASPLOS00[C], 190~201. [20] Leslie Lamport, Robert Shostak, Ma rshall Pease. The Byzantine Generals Problem[J]. ACM Transactions on Programming Languages and Systems, 1982.4(3): 382~401. [21] Sean Rhea, Patrick Eaton, Dennis Geels, et al. The OceanStore Prototype[A]. In Proc 2nd USENIX Conf on File and Storag e Technologies (FAST03)[C], 1~14. [22] Antony Rowstron, Peter Druschel. Storag e Management and Caching in PAST, a Large-scale, Persistent Peer-toPeer Storage Utility[A]. In SOSP01[C], 2001: 188~201 [23] Peter Druschel, Antony Rowstron. PAST: A large-scale, persistent peer-to-peer storage utility[A]. In Proc 8th ACM Symp on Operating Systems principles (HotOS01)[C], New York, ACM Press, 2001: 188~201 [24] Antony Rowstron, Peter Druschel. Past ry: Scalable, Decentralized Object Location and Routing for Large-scale Pe er-to-Peer Systems[A]. In Proc 18th IFIP/ACM Intl Conf on Distributed Sy stems Platforms (Middleware01)[C], LNCS 2218, Springer Verlag, 2001: 329~350. [25] . [M]. : , 2010: 1~9. [26] Sanjay Ghemawat, Howard Gobioff, Shun-Tak Leung. The Google File System[A]. In Proceedings of 19th ACM Symposium on Operating Systems Principles[C], 2003: 20~43. [27] Sun Lustre Networking: High-Performan ce Features and Fl exible Support f or a Wide Array of Netw orks[EB/OL]. https://www.sun.com/offers/details/lu stre_networking.xml. [28] Prabhakar Chaganti. Cloud computing w ith Amazon Web Services, Part 2: Storage in the cloud with Amazon Si mple Storage Serv ice(S3)[EB/OL], 200 8. http://www.ibm.com/devel operworks/liber/ar-cloudaws2/. [29] Prabhakar Chaganti. Cloud computing with Amazon We b Services, Part 1: I

106ntroduction[EB/OL], 2008. http://www .ibm.com/developerworks/liber/ar-clouda ws1/. [30] Scott Patten. The S3 Cookbook: Get cook ing with Amazons Simple Storage Service[M], Sopobo, 2009. [31] Amazon. Amazon S3 Developer Guide[ EB/OL], 2009. http://docs.amazonweb services.com -/AmazonS3/latest/. [32] D.Chappell. Introducting the Azure Se rvices[EB/OL], 2008.10. http://www.mi crosoft.com/azure/whitepaper.mspx. [33] D. Chappell. Introducti ng Windows Azure[EB/OL], 2009.01. http://www.micr osoft.com/azure/whitepaper.mspx. [34] Microsoft. Windows Azure Blob[EB/OL] , 2008.11. http://www.microsoft.com/ azure/whitepaper.mspx. [35] Microsoft. Windows Azure Table[EB/O L], 2008.11. http://www.microsoft.com /azure/whitepaper.mspx. [36] Microsoft. Windows Azure Queue[EB/OL] , 2008.11. http://www.microsoft.co m/azure/whitepaper.mspx. [37] Tom Wbite. Hadoop[M]. , . : , 2010.5: 44~79. [38] . -SaaS[M]. : , 2010.4: 202~209. [39] . 286[EB/OL]. 2010.9. http://stora ge.it168.com/ [40] . [D]. : , 2004: 3~5. [41] Mukesh Singhal, Niranjan G-shivaratri. A dvanced Concepts in Operating System, Distributed, Database, a nd Multiprocessor Operating Systems[M]. McGraw-Hid, INC, 1994. [42] G. Somasundaram, Alok Shrivastava. [M]. , , . : , 2010: 210~218. [43] . IP-SAN[D]. : , 2008: 2~4. [44] . [D]. : , 2004: 3~4. [45] A. Brinkman, M. Heidebuer, F. Meyer auf der Heide, et al. V: Drive-Costs and benefits of an out of band storag e virtualization system[A]. The 12th NASA Goddard 21st IEEE Conference on Mass Storage Systems and Technologies (MSST)[C], College Park, Maryland, USA, 2004.

107[46] File System in Userspace[EB/OL]. http://fuse.sourcef orge.net/, 2010.10. [47] Dokan, User mode file system for windows[EB/OL]. http://dokan-dev.net/en/, 2010.10. [48] , . RS[J]. , 2010.30(12): 3197-3200. [49] , , . . . 2010.11(4): 413~416. [50] H. Weatherspoon. J. Kubiatowicz. Erasure Coding Vs. Replication: A Quantitative Comparison[A]. IPTPS[C], 2002: 328-338. [51] R. W. Hamming. Error Detecting and E rror Correcting Codes[J]. The Bell System Technical Journal, 1950.4, Vol. XXIX, NO. 2. [52] L. Rizzo, and 1.Vicisano. A reliable multi cast data distributi on protocol based on software FEC Techniques[A]. In Pro ceeding of HPCS'97[C], Greece, 1997.6: 116-125. [53] L. Rizzo. DEIT Technical Report, LR-970131[R]. On the feasibility of software FEC, 1997. [54] J. Blomer, M. Mitzenmacher, and A. Shokr ollahi. An XOR-based erasure-resilient coding scheme, NO. TR-95048[R]. IC SI Technical Report, 1995.8. [55] Lacan and Fimes. Systematic MDS Erasure Codes based on Vandermonde Matrices[A]. IEEE Communicati ons Letters[C], 2004.9: 570~572. [56] . [M]. . 2007.6:31-36. [57] D. Coppersmith, S. Winograd. Matr ix multiplication via arithmetic progressions[A]. Proceedings of the 19th annual ACM symposium on Theory of computing[C], New York: ACM Press, 1987: 1-6. [58] Shokrollahi. Raptor codes, Technical Report DR2003-06-001[R]. Digital Fountain, 2003. [59] R. G. Gallager. Low Density Parity Check Codes[D]. MIT Press, Cambridge, 1963. [60] Xu S Y. On Dependability of Computing System[J]. Jour nal of Computer Science and Technology, 1999, 20(2): 198-206. [61] Anderson T, Laprie J C. Dependability: Basic Concep ts and Terminology[J]. ACM Trans. Dependable Computing and Fa ult Tolerance, 1990, 20(6): 123-131. [62] Rabinovich. M, Lazowska. E. D. An Ef ficient and Highly Available Read-one Write-All Protocol for Replicated Data Management[A]. Proceedings of the Second International Conference on Para llel and Distributed Information

108Systems[C]. San Diego, CA, 1993. 56-65. [63] IBM. http://www.research.ibm.com/autonomic/manifesto/autonomic_computing. pdf[EB/OL]. 2010.10. [64] J. Kephart, D. Chess. The Vision of Autonomic Computing[J]. IEEE Computer Society, 2003, 36(1): 41~59. [65] Roy Fielding. Architectural Styles and the Design of Network-based Software Architecture[D], Dept. of Co mputer Science, Univ. of California, Irvine, 2000 [66] R. L. Costello. Building Web Servic es the REST way[EB/OL]. http://www.x front.com/REST-Web-Services.html. [67] , , . Web[J]. , 2009,35(8): 85-87. [68] UDT: Breaking the Data Transfer Bottleneck[EB/OL]. http://udt.sourceforge.net/, Oct 12. [69] , , . UDT[J]. , 209.10: 148-151.

109 1 AC-RU (Adjust-Controlled Recently-Used) ADFS (Alibaba Distributed File System) AFS (Andrew File System) AVSG (Accessible Volume Storage Group) CFS (Chord File System) CIFS (Common Internet File System) DAS (Direct Access Storage) DoS (Denial of Service) FC (Fiber Channel) FCIP (Fiber Channel over IP) TCP/IP FFS (Formicary File System) FIFO (First In First Out) FTP (File Transfer Protocol) GB (Giga Byte) GF (Galois Field) GFS (Google File System) GIS (Geographic Information System) GUID (Global Unique Identifier) HBA (Host Bus Adaptor) HDFS (Hadoop Distributed File System) Hadoop HTTP (Hyper Text Transfer Protocol) IDA (Information Disperal Algorithm) IDC (International Data Corporation) IDC (Internet Data Center) IDE (Integrated Device Electronics) IOPS (Input/Output Per Second) / KB (Kilo Byte) LAN (Local Network Area) LRU (Least Recently Used) MB (Mega Byte) MBPS (Mega Byte Per Second) MPEG (Moving Pictures Experts Group) MTBF (Mean Time Between Failure) MTTR (Mean Time To Repair)

110NAS (Network Attached Storage) NFS (Network File System) OOA (Object Oriented Architecture) P2P (Peer to Peer) PAL (Phase Alternating Line) PB (Peta Byte) POA (Procedure Oriented Architecture) POSIX (Portable Operating System Interface of Unix) QoS (Quality of Service) RAID (Redundant Array of Independent Disk) REST (Repersentational State Transfer) ROA (Resource Oriented Architecture) RS (Reed-Solomon) - S3 (Simple Storage Service) SAN (Storage Area Network) SATA (Serial Advanced Technology Attachment) SCSI (Small Computer System Interface) SNIA (Storage Networking Industry Association) SOA (Service Oriented Architecture) TB (Trillion Byte) TFS (Tabao File System) UDT (UDP-Based Data Transfer) UDP VOD (Video On Demand) VSG (Volume Storage Group)

111 2

112