Concurrency (raqobatdoshlik/bir vaqtning o'zida ishlash) va parallelizm haqidagi uzoq nazariy ma'ruzalarni o'tkazib yuborib, to'g'ridan-to'g'ri Go tilida concurrent dastur yozishga sho'ng'iymiz!
Faraz qilaylik, bizda biror iborani so'zma-so'z, oralarida tasodifiy tanaffus (pauza) bilan ekranga chiqaruvchi funksiya bor:
package main
import (
"fmt"
"math/rand"
"strings"
"time"
)
// say iboradagi har bir so'zni ekranga chiqaradi.
func say(phrase string) {
for _, word := range strings.Fields(phrase) {
fmt.Printf("Simon says: %s...\n", word)
dur := time.Duration(rand.Intn(100)) * time.Millisecond
time.Sleep(dur)
}
}
func main() {
say("go is awesome")
}
Uni main funksiyasidan chaqiramiz:
Simon says: go...
Simon says: is...
Simon says: awesome...
Endi ikkita so'zlovchini yarataylik, har biri o'zining mustaqil iborasini aytsin:
package main
import (
"fmt"
"math/rand"
"strings"
"time"
)
// say har bir ishchining iborasini so'zma-so'z chiqaradi.
func say(id int, phrase string) {
for _, word := range strings.Fields(phrase) {
fmt.Printf("Worker #%d says: %s...\n", id, word)
dur := time.Duration(rand.Intn(100)) * time.Millisecond
time.Sleep(dur)
}
}
func main() {
say(1, "go is awesome")
say(2, "cats are cute")
}
Dasturni ishga tushirib ko'ramiz:
Worker #1 says: go...
Worker #1 says: is...
Worker #1 says: awesome...
Worker #2 says: cats...
Worker #2 says: are...
Worker #2 says: cute...
Yomon emas, ammo funksiyalar ketma-ket gapirdi: 1-ishchi o'z gapini to'liq tugatgachgina 2-ishchi boshladi. Ularning bir vaqtda (parallel/raqobatdosh ravishda) gapirishini ta'minlash uchun say() funksiyasini chaqirishdan oldin go kalit so'zini qo'shamiz:
func main() {
go say(1, "go is awesome")
go say(2, "cats are cute")
time.Sleep(500 * time.Millisecond)
}
Mana endi ular rostdan ham bizning e'tiborimiz uchun o'zaro raqobatlasha boshlaydi!
Worker #2 says: cats...
Worker #1 says: go...
Worker #2 says: are...
Worker #1 says: is...
Worker #2 says: cute...
Worker #1 says: awesome...
Biz go f() deb yozganimizda, f() funksiyasi boshqalardan mutlaqo mustaqil ravishda ishga tushadi.
[!TIP] Agar siz Python, JavaScript yoki boshqa tillardagi
async/awaitorqali asinxron dasturlash bilan tanish bo'lsangiz, u yerdagi tajribangizni to'g'ridan-to'g'ri Goga tatbiq qilishga urinmang. Go concurrency'ga tamoman boshqacha yondashadi. Unga yangi va toza nigoh bilan qarashga harakat qiling.
go kalit so'zi bilan ishga tushirilgan funksiyalar goroutine deb ataladi. Go runtime tizimi ushbu goroutine'larni boshqaradi va ularni protsessor (CPU) yadrolarida ishlayotgan operatsion tizim (OS) thread'lari bo'ylab aqlli taqsimlaydi. Operatsion tizimning og'ir thread'lari bilan solishtirganda, goroutine'lar juda yengil vaznli (atigi ~2KB boshlang'ich xotira stack'iga ega), shu sababli siz bir necha yuzlab, minglab, hatto o'n minglab goroutine'larni xotirjamlik bilan yaratishingiz mumkin.
Sizda tabiiy savol tug'ilishi mumkin: nega main funksiyasining oxiriga time.Sleep() yozdik? Keling, buni aniqlashtiraylik.
Goroutine'lar mutlaqo mustaqildir. Biz go say(...) deb chaqirganimizda, funksiya o'zicha mustaqil oqimda ishlay boshlaydi. main funksiyasi uning tugashini kutib o'tirmaydi.
Shuning uchun agar biz mainni quyidagicha yozsak:
func main() {
go say(1, "go is awesome")
go say(2, "cats are cute")
}
— dastur ekranga hech narsa chiqarmaydi! Sababi: main funksiyasi bizning goroutine'larimiz biror so'zni gapirishga ulgurmasidan oldin o'z ishini yakunlaydi. main o'z ishini tugatishi bilan butun dastur darhol to'xtatiladi.
main funksiyasining o'zi ham goroutine hisoblanadi, faqat u dastur ishga tushganda Go runtime tomonidan avtomatik start oladi. Demak, bizning kodimizda uchta goroutine bor: main, say(1) va say(2). Ularning barchasi bir-biridan mustaqil ishlaydi. Faqat bitta nozik tomoni bor: main goroutine tugashi bilan qolgan barcha goroutine'lar ham majburiy tarzda yopiladi.
Goroutine'larning tugashini kutish uchun time.Sleep() dan foydalanish juda yomon usul, chunki goroutine'lar o'z vazifasini qancha vaqtda bajarishini oldindan aniq taxmin qilib bo'lmaydi. Eng to'g'ri va ishonchli yondashuv — sync.WaitGroup (kutish guruhi)dan foydalanishdir:
package main
import (
"fmt"
"math/rand"
"strings"
"sync"
"time"
)
func main() {
var wg sync.WaitGroup // (1)
wg.Add(1) // (2)
go say(&wg, 1, "go is awesome")
wg.Add(1) // (2)
go say(&wg, 2, "cats are cute")
wg.Wait() // (3)
}
// say iboradagi har bir so'zni ekranga chiqaradi.
func say(wg *sync.WaitGroup, id int, phrase string) {
for _, word := range strings.Fields(phrase) {
fmt.Printf("Worker #%d says: %s...\n", id, word)
dur := time.Duration(rand.Intn(100)) * time.Millisecond
time.Sleep(dur)
}
wg.Done() // (4)
}
Bu qanday ishlaydi:
wg ➊ ichida hisoblagich (counter) bor.wg.Add(1) ➋ hisoblagichni bittaga oshiradi.wg.Done() ➍ hisoblagichni bittaga kamaytiradi.wg.Wait() ➌ hisoblagich nolga teng bo'lguncha joriy goroutine'ni (mainni) bloklab turadi.Shu tarzda main dasturdan chiqib ketmasdan oldin say(1) va say(2) to'liq tugashini kafolatli kutib turadi.
Biroq, bu yondashuv biznes mantig'ini (say) concurrency mantig'i (wg) bilan qorishtirib yuboradi. Natijada biz say funksiyasini oddiy, ketma-ket ishlaydigan kodda yoki boshqa joyda osonlikcha qayta ishlata olmaymiz.
Go tilida concurrency mantig'ini biznes mantig'idan ajratish tavsiya etiladi. Odatda bu anonim funksiyalar (closure) orqali amalga oshiriladi:
func main() {
var wg sync.WaitGroup
wg.Add(2)
go func() {
defer wg.Done()
say(1, "go is awesome")
}()
go func() {
defer wg.Done()
say(2, "cats are cute")
}()
wg.Wait()
}
Natija:
Worker #2 says: cats...
Worker #1 says: go...
Worker #1 says: is...
Worker #2 says: are...
Worker #2 says: cute...
Worker #1 says: awesome...
Bu mexanizmning afzalliklari:
wg.Add(2) deb chaqiramiz. (Xohlasak, har bir goroutine oldidan wg.Add(1) chaqirish ham mumkin — natija bir xil).go orqali alohida goroutine sifatida boshlanadi.defer wg.Done() goroutine qanday tugashidan qat'i nazar (hatto say ichida xatolik yoki panic yuz bersa ham) hisoblagichni kamaytirishni kafolatlaydi.say funksiyasining o'zi esa hech qanday WaitGroup haqida bilmaydi va oddiy toza funksiya bo'lib qoladi.Go 1.25 versiyasidan boshlab taqdim etilgan WaitGroup.Go metodi avtomatik ravishda kutish guruhi hisoblagichini oshiradi, ko'rsatilgan funksiyani alohida goroutine'da ishga tushiradi va u tugashi bilan hisoblagichni bittaga kamaytiradi.
Bu orqali yuqoridagi kodni wg.Add() va wg.Done() ni qo'lda yozmasdan ancha ixchamlashtirish mumkin:
package main
import (
"fmt"
"sync"
)
func main() {
var wg sync.WaitGroup
wg.Go(func() {
fmt.Println("go is awesome")
})
wg.Go(func() {
fmt.Println("cats are cute")
})
wg.Wait()
fmt.Println("done")
}
Uning Go standart kutubxonasidagi ichki tuzilishi aynan biz yuqorida ko'rganimizdek Add va Donega asoslangan:
// sync/waitgroup.go dan olingan:
func (wg *WaitGroup) Go(f func()) {
wg.Add(1)
go func() {
defer wg.Done()
f()
}()
}
Berilgan satrdagi raqamlar sonini sanovchi tayyor yordamchi funksiya:
// countDigits satrdagi raqamlar sonini qaytaradi.
func countDigits(str string) int {
count := 0
for _, char := range str {
if unicode.IsDigit(char) {
count++
}
}
return count
}
countDigitsInWords(phrase string) counter funksiyasini yozing. U berilgan iborani so'zlarga ajratishi va har bir so'zdagi raqamlarni alohida goroutine ichida countDigits orqali sanashi kerak.
Biz hali turli goroutine'lardan umumiy ma'lumotlarni o'zgartirish (mutex) mavzusini o'tmaganimiz sababli, bir nechta goroutine bir vaqtda xavfsiz yoza oladigan sync.Map obyekti — syncStatsdan foydalaniladi. Natijani saqlash uchun syncStats.Store(word, count) metodini chaqiring.
// countDigitsInWords iboradagi har bir so'zning raqamlari sonini hisoblaydi.
func countDigitsInWords(phrase string) counter {
var wg sync.WaitGroup
syncStats := new(sync.Map)
words := strings.Fields(phrase)
// TODO: Har bir so'z uchun alohida goroutine ishga tushiring,
// undagi raqamlarni sanang va natijani syncStats.Store(word, count) ga saqlang.
// Barcha goroutine'lar tugashini WaitGroup orqali kuting.
return asStats(syncStats)
}
func countDigitsInWords(phrase string) counter {
var wg sync.WaitGroup
syncStats := new(sync.Map)
words := strings.Fields(phrase)
for _, word := range words {
wg.Add(1)
go func(w string) {
defer wg.Done()
count := countDigits(w)
syncStats.Store(w, count)
}(word)
}
wg.Wait()
return asStats(syncStats)
}
[!NOTE] E'tibor bering, biz
wordqiymatini anonim funksiyaga argument (w string) sifatida uzatdik:go func(w string) { ... }(word). Bu sikl o'zgaruvchisining closure ichida to'g'ri o'qilishini ta'minlaydi. Barcha goroutine'lar ishini tugatgach (wg.Wait()), natijacounterformatida qaytariladi.
Bir nechta goroutine'ni ishga tushirish qulay, ammo ular o'zaro qanday ma'lumot almashadi?
Go tilida goroutine'lar bir-biriga qiymatlarni channel (kanal)lar orqali uzatadi. Channel xuddi bir goroutine biror narsani otib yuborsa, ikkinchi goroutine uni ilib oladigan maxsus aloqa tuynugiga o'xshaydi:
┌─────────────┐ ┌─────────────┐
│ Goroutine A │ │ Goroutine B │
│ ┌────┐ │ │ ┌────┐ │
│ X │ <- │ ───┼─ chan ◄─────┼── │ <- │ X │
│ └────┘ │ │ └────┘ │
└─────────────┘ └─────────────┘
Goroutine B goroutine A ga X qiymatini yubormoqda.
Channel qanday ishlaydi:
package main
import "fmt"
func main() {
// Channel yaratish uchun make(chan Tip) sintaksisi ishlatiladi.
// Channel faqat ko'rsatilgan tipdagi qiymatlarni qabul qila oladi:
messages := make(chan string)
// Channel'ga qiymat yuborish uchun `channel <-` sintaksisi ishlatiladi.
// "ping" xabarini yuboramiz:
go func() {
messages <- "ping"
}()
// Channel'dan qiymat qabul qilish uchun `<-channel` sintaksisi ishlatiladi.
// Xabarni qabul qilamiz va ekranga chiqaramiz:
msg := <-messages
fmt.Println(msg)
}
Dastur ishga tushganda, birinchi (anonim) goroutine ikkinchi (main) goroutine'ga messages kanali orqali "ping" xabarini uzatadi.
Channel orqali qiymat yuborish sinxron (bloklovchi) amaldir:
messages <- "ping"), u boshqa biror goroutine bu qiymatni qabul qilib olmagunicha (<-messages) to'xtab, kutib turadi.package main
import (
"fmt"
"time"
)
func main() {
messages := make(chan string)
go func() {
fmt.Println("B: Xabar yuborilmoqda...")
messages <- "ping" // (1)
fmt.Println("B: Xabar yuborildi!") // (2)
}()
fmt.Println("A: Qandaydir vazifa bajarilmoqda...")
time.Sleep(500 * time.Millisecond)
fmt.Println("A: Xabarni qabul qilishga tayyor...")
<-messages // (3)
fmt.Println("A: Xabar qabul qilindi!")
time.Sleep(100 * time.Millisecond)
}
Natija:
A: Qandaydir vazifa bajarilmoqda...
B: Xabar yuborilmoqda...
A: Xabarni qabul qilishga tayyor...
B: Xabar yuborildi!
A: Xabar qabul qilindi!
Dasturlashda ko'pincha "Producer-Consumer" (Ishlab chiqaruvchi — Iste'molchi) andozasini uchratamiz:
Keling, so'zlardagi raqamlarni sanash funksiyamizni channel orqali muloqot qiladigan qilib qayta yozamiz:
type counter map[string]int
countDigits bilan sanang va natijani counted kanaliga yozing (producer).countDigitsInWords) funksiyada kanaldan qiymatlarni o'qib olib, stats map'ini to'ldiring (consumer).sync.WaitGroup ishlatmang.func countDigitsInWords(phrase string) counter {
words := strings.Fields(phrase)
counted := make(chan int)
stats := counter{}
go func() {
// TODO: so'zlar bo'ylab sikl aylaning,
// har biridagi raqamlar sonini sanab counted kanaliga yozing.
}()
// TODO: counted kanalidan o'qib oling va stats xaritasini to'ldiring.
return stats
}
func countDigitsInWords(phrase string) counter {
words := strings.Fields(phrase)
counted := make(chan int)
stats := counter{}
go func() {
for _, word := range words {
counted <- countDigits(word)
}
}()
for _, word := range words {
stats[word] = <-counted
}
return stats
}
Hozirgacha biz countDigitsInWords funksiyasi barcha so'zlarni oldindan biladi deb hisobladik.
Biroq real loyihalarda ma'lumotlar ma'lumotlar bazasidan yoki tarmoq oqimidan qadamma-qadam kelishi mumkin va funksiya jami nechta so'z kelishini oldindan bilmaydi.
Bu holatni modellashtirish uchun ibora (satr) o'rniga next generator funksiyasini uzatamiz. next() har safar chaqirilganda navbatdagi so'zni beradi. So'zlar tugaganda esa bo'sh satr "" qaytaradi.
Oddiy, ketma-ket dastur quyidagicha ko'rinishga ega bo'ladi:
package main
import (
"fmt"
"strings"
"unicode"
)
type counter map[string]int
// countDigitsInWords so'zlardagi raqamlarni sanaydi,
// navbatdagi so'zni next() orqali oladi.
func countDigitsInWords(next func() string) counter {
stats := counter{}
for {
word := next()
if word == "" {
break
}
count := countDigits(word)
stats[word] = count
}
return stats
}
func countDigits(str string) int {
count := 0
for _, char := range str {
if unicode.IsDigit(char) {
count++
}
}
return count
}
func wordGenerator(phrase string) func() string {
words := strings.Fields(phrase)
idx := 0
return func() string {
if idx >= len(words) {
return ""
}
w := words[idx]
idx++
return w
}
}
func main() {
phrase := "0ne 1wo thr33 4068"
next := wordGenerator(phrase)
stats := countDigitsInWords(next)
for w, c := range stats {
fmt.Printf("%s: %d\n", w, c)
}
}
Agar biz generatorli misolimizni oldingi mashq kabi hal qilishga urinsak, ikkita qiyinchilikka duch kelamiz:
stats map'ining kaliti bo'lgan so'zni qayerdan olamiz?Ikkala muammoni hal qilish uchun kanalga faqat son (int) emas, balki so'z va sonni birlashtirgan juftlik — pair struct'ini yuborish lozim:
type pair struct {
word string
count int
}
counted := make(chan pair) kanalini oching.pair{"", 0} yuborib to'xtating.pair{word, countDigits(word)} jo'nating.statsni qaytaring.type pair struct {
word string
count int
}
func countDigitsInWords(next func() string) counter {
counted := make(chan pair)
go func() {
// TODO: next() dan so'zlarni olib, counted ga pair jo'nating
}()
// TODO: counted dan o'qib oling va stats ni to'ldiring
return stats
}
func countDigitsInWords(next func() string) counter {
counted := make(chan pair)
go func() {
for {
word := next()
if word == "" {
counted <- pair{"", 0}
break
}
counted <- pair{word, countDigits(word)}
}
}()
stats := counter{}
for {
p := <-counted
if p.word == "" {
break
}
stats[p.word] = p.count
}
return stats
}
Oldingi mashqda bitta goroutine so'zlarni o'qib, raqamlarini hisobladi:
┌───────────────┐ ┌────────────────┐
│ So'zlarni │ │ │
│ o'qiydi va │ → (counted) chan → stats'ni │
│ raqam sanaydi │ │ to'ldiradi │
└───────────────┘ └────────────────┘
goroutine tashqi funksiya
Haqiqiy katta loyihalarda ma'lumotlarni o'qish (Reader) bilan ularni hisoblash/qayta ishlash (Worker) vazifalarini alohida-alohida goroutine'larga ajratish ancha to'g'ri va samarali arxitekturadir:
┌───────────────┐ ┌───────────────┐ ┌────────────────┐
│ So'zlarni │ │ Raqamlarni │ │ │
│ o'qib │ → (pending) chan → hisoblaydi │ → (counted) chan → stats'ni │
│ uzatadi │ │ │ │ to'ldiradi │
└───────────────┘ └───────────────┘ └────────────────┘
reader goroutine worker goroutine tashqi funksiya
pending := make(chan string) va counted := make(chan pair) kanallarini oching.pending kanaliga yuboradi (tugaganda "" yuborib chiqadi).pending kanalidan so'zlarni o'qiydi, har bir so'zdagi raqamlarni sanaydi va natijani pair ko'rinishida counted kanaliga jo'natadi (bo'sh so'z kelganda pair{"", 0} yuborib chiqadi).counted kanalidan qabul qilib, yakuniy stats map'ini to'ldiradi.func countDigitsInWords(next func() string) counter {
pending := make(chan string)
counted := make(chan pair)
// Reader goroutine
go func() {
for {
word := next()
pending <- word
if word == "" {
break
}
}
}()
// Worker goroutine
go func() {
for {
word := <-pending
if word == "" {
counted <- pair{"", 0}
break
}
counted <- pair{word, countDigits(word)}
}
}()
// Consumer (tashqi funksiya)
stats := counter{}
for {
p := <-counted
if p.word == "" {
break
}
stats[p.word] = p.count
}
return stats
}
Mantiq Reader va Worker o'rtasida bo'lingach, countDigitsInWords funksiyasi kattalashib ketdi. Unda uchta aniq mantiqiy qadam bor:
submitWords).countWords).fillStats).Bu bloklarni alohida nomlangan funksiyalarga ajratish kodni o'qishni juda osonlashtiradi:
func countDigitsInWords(next func() string) counter {
pending := make(chan string)
go submitWords(next, pending)
counted := make(chan pair)
go countWords(pending, counted)
return fillStats(counted)
}
Yuqoridagi arxitekturaga mos keladigan submitWords, countWords va fillStats funksiyalarini yozing:
// submitWords generatordan o'qib, so'zlarni kanalga yuboradi.
func submitWords(next func() string, pending chan string) {
for {
word := next()
pending <- word
if word == "" {
break
}
}
}
// countWords pending kanalidan o'qib, hisoblab, counted kanaliga uzatadi.
func countWords(pending chan string, counted chan pair) {
for {
word := <-pending
if word == "" {
counted <- pair{"", 0}
break
}
counted <- pair{word, countDigits(word)}
}
}
// fillStats kanaldan barcha natijalarni o'qib, map tayyorlaydi.
func fillStats(counted chan pair) counter {
stats := counter{}
for {
p := <-counted
if p.word == "" {
break
}
stats[p.word] = p.count
}
return stats
}
Biz yuqorida erishgan funksiya:
func countDigitsInWords(next func() string) counter {
pending := make(chan string)
go submitWords(next, pending)
counted := make(chan pair)
go countWords(pending, counted)
return fillStats(counted)
}
Kanalni submitWordsning o'zida yaratib, ota funksiyaga qaytarish yaxshiroq emasmi? Shunda kanalga to'liq submitWords egalik qiladi (channel ownership). Xuddi shuningdek, counted kanaliga ham countWords egalik qiladi.
U holda countDigitsInWords quyidagicha toza ko'rinishga keladi:
func countDigitsInWords(next func() string) counter {
pending := submitWords(next)
counted := countWords(pending)
return fillStats(counted)
}
Endi kanallarga egalik huquqi juda aniq. Lekin goroutine'lar qayerga ketdi? Biz goroutine'larni aynan submitWords va countWords funksiyalari ichida ishga tushiramiz:
// submitWords so'zlarni sanash uchun yuboradi va chiqish kanalini qaytaradi.
func submitWords(next func() string) chan string {
out := make(chan string)
go func() {
for {
word := next()
out <- word
if word == "" {
break
}
}
}()
return out
}
// countWords so'zlardagi raqamlarni hisoblaydi va chiqish kanalini qaytaradi.
func countWords(in chan string) chan pair {
out := make(chan pair)
go func() {
for {
word := <-in
if word == "" {
out <- pair{"", 0}
break
}
out <- pair{word, countDigits(word)}
}
}()
return out
}
To'liq dasturni ishga tushirib ko'ramiz:
package main
import (
"fmt"
"strings"
"unicode"
)
type counter map[string]int
type pair struct {
word string
count int
}
func countDigits(str string) int {
count := 0
for _, char := range str {
if unicode.IsDigit(char) {
count++
}
}
return count
}
func wordGenerator(phrase string) func() string {
words := strings.Fields(phrase)
idx := 0
return func() string {
if idx >= len(words) {
return ""
}
w := words[idx]
idx++
return w
}
}
func submitWords(next func() string) chan string {
out := make(chan string)
go func() {
for {
word := next()
out <- word
if word == "" {
break
}
}
}()
return out
}
func countWords(in chan string) chan pair {
out := make(chan pair)
go func() {
for {
word := <-in
if word == "" {
out <- pair{"", 0}
break
}
out <- pair{word, countDigits(word)}
}
}()
return out
}
func fillStats(counted chan pair) counter {
stats := counter{}
for {
p := <-counted
if p.word == "" {
break
}
stats[p.word] = p.count
}
return stats
}
func countDigitsInWords(next func() string) counter {
pending := submitWords(next)
counted := countWords(pending)
return fillStats(counted)
}
func main() {
phrase := "0ne 1wo thr33 4068"
next := wordGenerator(phrase)
stats := countDigitsInWords(next)
for word, count := range stats {
fmt.Printf("%s: %d\n", word, count)
}
}
Natija:
0ne: 1
1wo: 1
thr33: 2
4068: 4
[!TIP] Funksiyadan chiqish kanalini (output channel) qaytarish va uni ichki goroutine orqali to'ldirish — Go tilida juda keng qo'llaniladigan va qulay konveyer (pipeline) andozalaridan biridir.
Ushbu bobda biz Go tilida goroutine'lar va channel'larning poydevori bilan amaliy tanishdik:
go kalit so'zi bilan goroutine ishga tushirish;main goroutine va mustaqil goroutine'lar o'rtasidagi munosabat;sync.WaitGroup va yangi wg.Go() yordamida goroutine'lar tugashini kutish;chan (kanallar) orqali xavfsiz ma'lumot uzatish va o'zaro sinxronizatsiya;Keyingi qadamda 02 Channels, 03 Pipelines va 04 Time mavzulari orqali bilimlaringizni chuqurlashtirib borishingiz mumkin!