Ib qho kev faib tawm hauv nws cov ntsiab lus yooj yim tshaj plaws yog ib pawg ntawm cov khoos phis tawj ua haujlwm ua ke uas tshwm sim ua ib tus neeg siv kawg. Cov cav tov sib koom ib lub xeev, khiav concurrently, thiab muaj peev xwm ua hauj lwm ntawm nws tus kheej yam tsis muaj kev cuam tshuam lub uptime ntawm tag nrho cov system. Qhov tseeb yog tias kev tswj cov kab ke no yog lub ntsiab lus nyuaj uas muaj qhov tsis zoo.
Txheej txheem cej luam ntawm qhov system
Qhov kev faib tawm tso cai rau kev sib koom ntawm cov peev txheej (nrog rau software) txuas nrog lub network tib lub sijhawm.
Piv txwv ntawm kev faib khoom:
- Tseem ceeb pawg. Cov ntaub ntawv no tau khaws cia rau hauv cov ntaub ntawv kaw lus ntawm ib lub tshuab. Thaum twg tus neeg siv xav tau cov ntaub ntawv, nws sib txuas lus ncaj qha nrog lub tshuab no. Txhawm rau faib cov ntaub ntawv no, koj yuav tsum khiav nws ntawm ntau lub PC tib lub sijhawm.
- Distributed architecture.
Distributed systemtso cai rau koj mus teev horizontally thiab vertically. Piv txwv li, tib txoj hauv kev los daws cov tsheb ntau dua yuav yog txhawm rau txhim kho cov khoom siv uas khiav cov ntaub ntawv. Qhov no yog hu ua ntsug scaling. Vertical scaling yog qhov zoo mus txog rau qee qhov kev txwv, tom qab ntawd txawm tias cov cuab yeej zoo tshaj plaws tsis tuaj yeem tiv nrog kev muab cov tsheb uas yuav tsum tau ua.
Scaling kab rov tav txhais tau tias ntxiv cov khoos phis tawj ntxiv, tsis hloov kho kho vajtse ntawm ib qho. Vertical scaling nce kev ua tau zoo rau cov cuab yeej kho vajtse tshiab kawg hauv cov tshuab faib. Cov cib fim no tsis txaus rau cov tuam txhab thev naus laus zis uas muaj cov haujlwm nruab nrab mus rau hnyav. Qhov zoo tshaj plaws txog kev ntsuas kab rov tav yog tias tsis muaj qhov txwv loj. Thaum kev ua haujlwm tsis zoo, lwm lub tshuab tsuas yog ntxiv, uas, hauv paus ntsiab lus, tuaj yeem ua tiav tas li.
Ntawm cov neeg koom tes, ib qho kev tswj hwm kev faib tawm feem ntau suav nrog ntau cov kauj ruam. Nyob rau hauv cov txheej txheem kev lag luam nyob rau hauv qhov chaw zoo tshaj plaws ntawm lub lag luam computer network. Piv txwv li, nyob rau hauv ib tug raug tis siv ib tug peb-tier faib system qauv, cov ntaub ntawv ua yog ua nyob rau hauv ib tug PC ntawm tus neeg siv qhov chaw, kev ua lag luam yog ua nyob rau hauv ib tug tej thaj chaw deb computer, thiab cov ntaub ntawv nkag mus thiab cov ntaub ntawv ua yog ua nyob rau hauv ib tug sib txawv kiag li lub computer. uas muab kev nkag mus rau hauv nruab nrab rau ntau lub lag luam. Feem ntau, hom kev faib xam nosiv tus qauv kev sib cuam tshuam tus neeg siv khoom-neeg rau zaub mov.
Main Tasks
Lub luag haujlwm tseem ceeb ntawm kev tswj hwm kev faib tawm suav nrog:
- Transparency - Ua tiav ib daim duab kab ke yam tsis muaj qhov zais qhov chaw, nkag mus, tsiv teb tsaws, sib koom ua ke, tsis ua haujlwm, hloov chaw, tsis tu ncua thiab cov ntsiab lus rau cov neeg siv.
- Qhib - ua kom yooj yim lub network teeb tsa thiab hloov pauv.
- Kev ntseeg tau - Piv rau ib qho kev tswj hwm ib leeg, nws yuav tsum txhim khu kev qha, zoo ib yam thiab muaj qhov ua tau zoo ntawm kev npog qhov yuam kev.
- Kev Ua Haujlwm - Piv nrog rau lwm cov qauv, cov qauv faib muab kev txhawb nqa.
- Scalable - Cov kev faib tswj cov txheej txheem no yuav tsum muaj peev xwm ua tau raws li thaj chaw, kev tswj hwm, lossis qhov loj.
Cov dej num ntawm kev faib khoom suav nrog:
- Kev nyab xeeb yog qhov teeb meem loj hauv ib puag ncig kev faib tawm, tshwj xeeb tshaj yog thaum siv cov kev sib txuas lus pej xeem.
- Kev ua siab ntev - tuaj yeem tawv thaum tus qauv tsim nrog cov khoom tsis ntseeg siab.
- Kev sib koom tes thiab kev faib cov peev txheej - tuaj yeem nyuaj yog tias tsis muaj cov txheej txheem tsim nyog lossis cov cai xav tau.
Distributed xam ib puag ncig
(DCE) yog tus qauv siv dav hauv kev lag luam uas txhawb nqa xws li kev faib xam. Hauv Is Taws Nem, cov neeg muab kev pabcuam thib peb muab qee qhov kev pabcuam dav dav,uas haum rau tus qauv no.
Grid suav yog cov qauv kev suav nrog cov qauv sib faib ntawm ntau lub khoos phis tawj cuam tshuam nrog kev daws teeb meem nyuaj. Nyob rau hauv daim phiaj xam qauv, cov servers lossis cov khoos phis tawj ntawm tus kheej ua haujlwm ywj pheej thiab txuas nrog ib leeg los ntawm Is Taws Nem lossis cov tes hauj lwm ceev ceev.
Qhov phiaj xwm loj tshaj plaws yog SETI@home, uas tus tswv lub khoos phis tawj tuaj yeem ua haujlwm rau qee qhov kev ua haujlwm ntau yam uas siv lawv lub khoos phis tawj rau Kev Tshawb Fawb Txog Extraterrestrial Intelligence (SETI) qhov project. Qhov teeb meem hauv computer no siv ntau txhiab lub khoos phis tawj los rub tawm thiab tshawb xyuas cov ntaub ntawv xov tooj cua telescope.
Ib qho ntawm thawj qhov kev siv ntawm daim phiaj suav yog txhawm rau txhawm rau cryptographic code los ntawm ib pab pawg tam sim no hu ua distribution.net. Pab pawg no kuj piav txog lawv cov qauv raws li kev faib xam.
Database scaling
Kev nthuav tawm cov ntaub ntawv tshiab ntawm tus tswv mus rau qhev tsis tshwm sim tam sim ntawd. Qhov tseeb, muaj lub sijhawm qhov rai uas koj tuaj yeem tau txais cov ntaub ntawv tsis tu ncua. Yog tias qhov no tsis yog qhov teeb meem, kev sau ntawv yuav raug kev txom nyem, vim tias cov tshuab xa tawm yuav tsum tau tos synchronously rau cov ntaub ntawv nthuav tawm. Lawv tuaj nrog ob peb qhov kev pom zoo.
Siv ib tug qhev database mus kom ze, nws muaj peev xwm mus scale tawm nyeem tsheb mus rau ib co kev. Muaj ntau txoj kev xaiv ntawm no. Tab sis koj tsuas yog yuav tsum tau faib cov ntawv sau rau hauv ob pebservers vim nws tsis tuaj yeem tuav nws. Ib txoj hauv kev yog siv ntau lub tswv yim rov ua dua tshiab. Nyob ntawd, tsis yog qhev, muaj ob peb lub ntsiab lus uas txhawb kev nyeem ntawv thiab sau ntawv.
Lwm txoj kev hu ua sharding. Nrog nws, tus neeg rau zaub mov tau faib ua ob peb lub servers me, hu ua shards. Cov shards no muaj qhov sib txawv, cov cai yog tsim los ntawm cov ntawv nkag mus rau qhov twg shard. Nws yog ib qho tseem ceeb heev los tsim ib txoj cai zoo li no tias cov ntaub ntawv raug faib sib npaug. Ib txoj hauv kev ua tau rau qhov no yog txhawm rau txheeb xyuas ntau yam raws li qee cov ntaub ntawv sau tseg.
Tus yuam sij shard no yuav tsum tau xaiv ua tib zoo, vim tias lub nra hnyav tsis tas yuav sib npaug ntawm cov hauv paus ntawm cov kab ke. Cov shard nkaus xwb uas tau txais kev thov ntau dua li lwm tus hu ua hotspot, thiab lawv sim tiv thaiv nws los ntawm kev tsim. Thaum sib cais, cov ntaub ntawv rov ua dua yuav kim heev thiab tuaj yeem ua rau muaj kev poob qis.
Database pom zoo algorithms
DBs nyuaj rau kev siv hauv kev faib kev nyab xeeb vim lawv xav kom txhua tus ntawm kev sib tham txog qhov raug cuam tshuam lossis ua txhaum. Qhov zoo no yog lub npe hu ua kev pom zoo thiab yog qhov teeb meem tseem ceeb hauv kev tsim kom muaj kev faib khoom. Kev ua tiav hom kev pom zoo xav tau rau qhov teeb meem "kev cog lus" yog qhov yooj yim yog tias cov txheej txheem cuam tshuam thiab lub network muaj kev ntseeg siab. Txawm li cas los xij, cov tshuab tiag tiag yuav raug rau ntau tus lejtuaj yeem ua tsis tiav ntawm cov txheej txheem kev sib txuas, ploj, cov lus tsis raug lossis luam tawm.
Qhov no ua rau muaj teeb meem thiab tsis tuaj yeem lav tias qhov kev pom zoo yuav raug mus txog hauv lub sijhawm txwv ntawm lub network tsis ntseeg. Hauv kev xyaum, muaj cov algorithms uas ncav cuag kev pom zoo sai sai hauv lub network tsis ntseeg. Cassandra yeej muab kev lag luam sib sib zog nqus los ntawm kev siv Paxos algorithm rau kev pom zoo sib faib.
Kev faib xam yog tus yuam sij rau kev nkag mus ntawm cov ntaub ntawv loj uas tau siv nyob rau xyoo tas los no. Nws yog ib txoj hauv kev los rhuav tshem ib txoj haujlwm loj, xws li cov ntaub ntawv khaws tseg 100 billion, uas tsis muaj ib lub computer muaj peev xwm ua tau txhua yam ntawm nws tus kheej, ua ntau yam haujlwm me me uas tuaj yeem haum rau hauv ib lub tshuab. Tus tsim tawm rhuav tshem nws txoj haujlwm loj rau ntau qhov me me, tua lawv ntawm ntau lub tshuab ua ke, sau cov ntaub ntawv kom tsim nyog, ces qhov teeb meem qub yuav raug daws.
Txoj hauv kev no tso cai rau koj los ntsuas kab rov tav - thaum muaj txoj haujlwm loj, tsuas yog ntxiv cov nodes rau kev suav. Cov haujlwm no tau ua tau ntau xyoo los ntawm MapReduce cov qauv kev ua haujlwm cuam tshuam nrog kev siv rau kev ua haujlwm sib luag thiab tsim cov ntaub ntawv loj uas siv cov txheej txheem faib rau ntawm pawg.
Tam sim no, MapReduce yog qhov qub thiab coj qee yam teeb meem. Lwm cov architectures tau tshwm sim uas hais txog cov teeb meem no. Namely, Lambda Architecture rau faibflow processing systems. Kev nce qib hauv cheeb tsam no tau coj cov cuab yeej tshiab: Kafka Stream, Apache Spark, Apache Storm, Apache Samza.
Cov ntaub ntawv khaws cia thiab rov ua dua tshiab
Cov ntaub ntawv faib tawm tuaj yeem xav tias yog cov khw muag cov ntaub ntawv faib. Qhov no yog tib yam li lub tswv yim - khaws cia thiab nkag mus rau ntau cov ntaub ntawv thoob plaws ib pawg ntawm cov tshuab uas yog ib qho chaw. Lawv feem ntau mus koom tes nrog Distributed Computing.
Piv txwv li, Yahoo tau paub txog kev khiav HDFS ntawm ntau dua 42,000 nodes los khaws 600 petabytes ntawm cov ntaub ntawv txij li xyoo 2011. Wikipedia txhais qhov sib txawv ntawm cov ntaub ntawv faib tawm tso cai rau cov ntaub ntawv nkag tau siv tib qhov sib cuam tshuam thiab cov ntsiab lus raws li cov ntaub ntawv hauv zos, tsis yog los ntawm API kev cai xws li Cassandra Query Language (CQL).
Hadoop Distributed File System (HDFS) yog ib qho system siv rau kev suav hla Hadoop infrastructure. Dav dav, nws yog siv los khaws thiab rov ua cov ntaub ntawv loj (GB lossis TB loj) ntawm ntau lub tshuab. Nws architecture muaj feem ntau ntawm NameNodes thiab DataNodes.
NameNodes yog lub luag haujlwm khaws cia metadata txog pawg, xws li cov node twg muaj cov ntaub ntawv thaiv. Lawv ua raws li kev sib koom tes hauv network, txheeb xyuas qhov zoo tshaj plaws los khaws thiab luam cov ntaub ntawv, ua raws li kev noj qab haus huv ntawm lub cev. DataNodes tsuas khaws cov ntaub ntawv thiab ua cov lus txib xws li cov ntaub ntawv rov ua dua, sau ntawv tshiab, thiablwm tus.
Unsurprisingly, HDFS yog qhov zoo tshaj plaws siv nrog Hadoop rau xam, raws li nws muab kev paub txog cov ntaub ntawv ua hauj lwm. Cov haujlwm tshwj xeeb yog tom qab ntawd khiav ntawm cov nodes uas khaws cov ntaub ntawv. Qhov no tso cai rau koj siv qhov chaw ntawm cov ntaub ntawv - optimizes kev suav thiab txo cov tsheb khiav hauv lub network.
Lub Interplanetary File System (IPFS) yog qhov zoo siab tshiab rau cov phooj ywg-rau-peer raws tu qauv/network rau kev faib cov ntaub ntawv. Siv Blockchain thev naus laus zis, nws khav theeb tag nrho cov qauv kev faib tawm uas tsis muaj ib tus tswv lossis qhov tsis ua tiav.
IPFS muaj cov npe npe (zoo ib yam li DNS) hu ua IPNS thiab tso cai rau cov neeg siv tau yooj yim khaws cov ntaub ntawv. Nws khaws cov ntaub ntawv los ntawm keeb kwm versioning, zoo li Git ua. Qhov no tso cai rau nkag mus rau tag nrho cov xeev yav dhau los ntawm cov ntaub ntawv. Nws tseem tab tom dhau los ntawm kev txhim kho hnyav (v0.4 thaum lub sijhawm sau ntawv) tab sis twb pom cov haujlwm txaus siab los tsim nws (FileCoin).
Messaging system
Cov kab lus xa xov muab qhov chaw nruab nrab rau khaws cia thiab xa cov lus hauv ib qho kev sib koom. Lawv tso cai rau koj cais daim ntawv thov kev xav los ntawm kev sib txuas lus ncaj qha nrog lwm lub tshuab.
Known scale - LinkedIn's Kafka pawg ua tiav 1 trillion cov lus hauv ib hnub nrog qhov siab tshaj ntawm 4.5 lab cov lus ib ob.
Hauv cov ntsiab lus yooj yim, lub platform xa ntawv ua haujlwm zoo li no:
- Messagedhau los ntawm daim ntawv thov uas muaj peev xwm tsim nws, hu ua tus tsim khoom, nkag mus rau hauv lub platform, thiab tau nyeem los ntawm ntau daim ntawv thov, hu ua cov neeg siv khoom.
- Yog tias koj xav tau khaws qee qhov xwm txheej hauv ntau qhov chaw, xws li tsim tus neeg siv rau lub chaw khaws ntaub ntawv, khaws cia, kev pabcuam xa email, ces lub platform xa xov yog qhov huv tshaj plaws los faib cov lus ntawd.
Muaj ntau lub xov tooj nrov tshaj plaws.
RabbitMQ yog tus xa xov xov xwm uas tso cai rau koj los kho kom zoo dua kev tswj hwm ntawm lawv txoj hauv kev uas siv cov cai routing thiab lwm yam kev teeb tsa yooj yim. Nws tuaj yeem raug hu ua "smart" broker vim nws muaj ntau qhov laj thawj thiab ua tib zoo saib xyuas cov lus uas dhau los ntawm nws. Muab kev xaiv rau APs thiab CPs los ntawm CAP.
Kafka yog tus xa xov xov xwm uas ua haujlwm tsawg dua vim nws tsis taug qab cov lus twg tau nyeem thiab tsis tso cai rau txoj hauv kev nyuaj. Nws pab ua tiav qhov kev ua tau zoo thiab sawv cev rau cov lus cog tseg loj tshaj plaws hauv qhov chaw no nrog kev txhim kho nquag ntawm cov kab ke sib faib los ntawm cov zej zog qhib thiab kev txhawb nqa ntawm pab pawg Confluent. Kafka yog nrov tshaj plaws nrog cov tuam txhab high-tech.
Machine Interaction Applications
Qhov kev faib tawm no yog ib pawg ntawm cov khoos phis tawj ua haujlwm ua ke kom tshwm sim raws li lub khoos phis tawj cais rau cov neeg siv kawg. Cov tshuab no nyob rau hauv cov xwm txheej, ua haujlwmib txhij thiab tuaj yeem ua haujlwm ntawm nws tus kheej yam tsis cuam tshuam rau lub sijhawm ua haujlwm ntawm tag nrho cov kab ke.
Yog tias koj xav txog cov ntaub ntawv raws li kev faib tawm, tsuas yog cov nodes cuam tshuam nrog ib leeg los tswj hwm lawv cov haujlwm. Nws yog nyob rau hauv cov ntaub ntawv no ib yam dab tsi zoo li daim ntawv thov khiav nws cov cai sab hauv ntawm ib tug phooj ywg-rau-peer network thiab muab cais raws li ib tug faib daim ntawv thov.
Piv txwv ntawm cov ntawv thov no:
- Known Scale - BitTorrent swarm 193,000 nodes rau Game of Thrones rov.
- Basic register technology ntawm faib Blockchain systems.
Cov ntaub ntawv faib tawm tuaj yeem xav tias yog qhov hloov tsis tau, daim ntawv thov-tsuas yog cov ntaub ntawv uas tau luam tawm, ua kom sib txuas, thiab sib koom thoob plaws txhua qhov ntawm cov khoom sib faib.
Cov nplais paub zoo - Ethereum network - muaj 4.3 lab kev hloov pauv ib hnub rau Lub Ib Hlis 4, 2018. Lawv siv tus qauv Kev Tshawb Fawb, uas tso cai rau koj los kho lub xeev ntawm cov ntaub ntawv txhua lub sijhawm.
Blockchain yog cov cuab yeej siv tam sim no siv rau cov ntaub ntawv faib khoom thiab tau cim lawv qhov pib. Qhov kev tsim kho tshiab tshaj plaws thiab loj tshaj plaws hauv qhov chaw faib khoom tsim thawj zaug kev them nyiaj raws tu qauv, bitcoin.
Blockchain yog ib daim ntawv faib khoom nrog cov npe xaj ntawm txhua qhov kev lag luam uas tau tshwm sim hauv nws lub network. Deals yog pab pawg thiab khaws cia hauv cov blocks. Tag nrho blockchain yog qhov tseem ceeb ntawm cov npe sib txuas ntawm cov blocks. Tshwj xeeb Blocksyog kim los tsim thiab nruj ua ke rau ib leeg los ntawm cryptography. Yooj yim muab tso, txhua qhov thaiv muaj qhov tshwj xeeb hash (uas pib nrog X tus lej ntawm xoom) ntawm cov ntsiab lus ntawm cov thaiv tam sim no (nyob rau hauv daim ntawv ntawm tsob ntoo Merkle) ntxiv rau cov hash ntawm lub block dhau los. Qhov hash no yuav tsum muaj ntau lub zog CPU.
Piv txwv ntawm kev faib ua haujlwm
System hom tshwm sim rau cov neeg siv vim lawv yog tib neeg siv tshuab. Lawv qhia lawv lub cim xeeb, disk, thiab tus neeg siv tsis muaj teeb meem navigating los ntawm cov ntaub ntawv. Tus neeg siv khaws qee yam hauv nws lub PC thiab cov ntaub ntawv khaws cia hauv ntau qhov chaw xws li cov khoos phis tawj txuas nrog kom cov ntaub ntawv ploj tuaj yeem rov qab tau yooj yim.
Piv txwv ntawm kev faib ua haujlwm:
- Windows Server 2003;
- Windows Server 2008;
- Windows Server 2012;
- UbuntuLinux (Apache server).
Yog tias ib lub khoos phis tawj khau raj siab dua, uas yog, yog tias muaj ntau qhov kev thov sib pauv ntawm ib tus PCs, qhov no yog qhov sib npaug li cas tshwm sim. Hauv qhov no, cov lus thov raug nthuav tawm mus rau lub PC nyob sib ze. Yog hais tias lub network ua ntau loaded, ces nws tuaj yeem nthuav dav los ntawm kev ntxiv ntau lub tshuab rau lub network. Cov ntaub ntawv network thiab cov folders yog synchronized thiab naming conventions yog siv kom tsis txhob yuam kev thaum cov ntaub ntawv yog retrieved.
Caching kuj tseem siv thaum tswj cov ntaub ntawv. Txhua lub khoos phis tawj siv tib lub npe chaw rau npe cov ntaub ntawv. Tab siscov ntaub ntawv system siv tau rau txhua lub computer. Yog tias muaj cov ntaub ntawv hloov tshiab, nws tau sau rau hauv ib lub computer thiab cov kev hloov pauv tau nthuav tawm mus rau txhua lub khoos phis tawj, yog li cov ntaub ntawv zoo ib yam.
Cov ntaub ntawv raug kaw thaum lub sijhawm nyeem / sau ntawv, yog li tsis muaj kev cuam tshuam ntawm cov khoos phis tawj sib txawv. Kev sib tham kuj tshwm sim, xws li kev nyeem ntawv, sau cov ntaub ntawv hauv ib ntu thiab kaw qhov kev sib tham, thiab tom qab ntawd lwm tus neeg siv tuaj yeem ua tau zoo ib yam thiab lwm yam.
txiaj ntsig ntawm kev siv
Ib qho kev ua haujlwm tsim los ua kom tib neeg lub neej txhua hnub yooj yim dua. Rau cov neeg siv cov txiaj ntsig thiab kev xav tau, lub operating system tuaj yeem yog ib tus neeg siv lossis faib. Hauv kev faib khoom siv, ntau lub khoos phis tawj txuas nrog rau ib leeg thiab faib lawv cov peev txheej.
Txoj txiaj ntsig ntawm kev ua qhov no:
- Yog tias ib lub PC hauv qhov system ua haujlwm tsis raug lossis puas, ces lwm lub ntawm lossis lub khoos phis tawj yuav saib xyuas nws.
- Ntau cov peev txheej tuaj yeem ntxiv tau yooj yim.
- Cov peev txheej xws li tshuab luam ntawv tuaj yeem ua haujlwm rau ntau lub khoos phis tawj.
Nov yog luv luv txog qhov kev faib tawm, vim li cas nws thiaj siv. Qee yam tseem ceeb uas yuav tsum nco ntsoov: lawv yog qhov nyuaj thiab raug xaiv rau qhov ntsuas thiab tus nqi thiab nyuaj rau kev ua haujlwm nrog. Cov tshuab no tau muab faib ua ntau pawg cia: suav, cov ntaub ntawv thiab xa xov, sau npe, daim ntawv thov. Thiab tag nrho cov no tsuas yog superficial heev txog cov ntaub ntawv nyuaj.
