# Multithreading in Julia

# Overview

Multithreading is the parallel processing, that is, several independent processes runs at the same time. This can be implementable on a multi-core CPU and is usually possible in any programming language.

The multithreading is avalable in Julia from v1.3.0-alpha, it can used for it specifing the number of logical cores(=threads).

After all, distributing the process for each thread have a few of overheads, thereby if each process is too small, the overall process time will be late in inverse because the overhead is a burden.

# Requirements

Multi core CPU
Julia v1.3.0-alpha

# Main

# How to check the number of cores

mac
- $ sysctl -n hw.logicalcpu
linux
- $ fgrep 'processor' /proc/cpuinfo | wc -l

# Settings

mac

$ echo "export JULIA_NUM_THREADS=`sysctl -n hw.logicalcpu`" >> ~/.zshrc

linux

$ THREADS=`fgrep 'processor' /proc/cpuinfo | wc -l
$ echo "export JULIA_NUM_THREADS=$THREADS" >> ~/.bashrc

The settings is as the above, and start-up or run Julia as below.

$ julia hogehuga.jl [あなたの素敵な引数] -u auto   # hogehuga.jlを実行
$ julia -t auto                                 # .zshrc(.bashrc)に記載されたthread数を起動
$ julia --threads 4                             # 指定したthread数を起動
$ julia -t 4                                    # 指定したthread数を起動
$ JULIA_NUM_THREADS=4 julia                     # 指定したthread数を起動

You can see whether Julia's session is for multithreading or not, do as below

julia> Threads.nthreads()
4

like above, the number of threads shows that.

# script

Thread package included in Base is for Multithreading in Julia

# Multithreading

function main()
    println("Thread数: ", Threads.nthreads())
    Threads.@threads for i = 1:20
        sleep(1)
        println(i)
    end
end

p, t = @timed main()
println("実行時間: ", t)

$ julia -t 8
Thread数: 8
7
13
10
... (中略)
9
12
3
6
(value = nothing, time = 3.072258373, bytes = 3019730, gctime = 0.0, gcstats = Base.GC_Diff(3019730, 0, 0, 54596, 14, 0, 0, 0, 0))

> 実行時間: 3.072258373

# Sequential processing

function main()
    for i = 1:20
        sleep(1)
        println(i)
    end
end

p, t = @timed main()
println("実行時間: ", t)

$ julia
1
2
3
... (中略)
19
20
(value = nothing, time = 20.103233506, bytes = 695522, gctime = 0.0, gcstats = Base.GC_Diff(695522, 0, 0, 13384, 1, 0, 0, 0, 0))
> 実行時間: 20.103233506

# summarize

According to Amdahl's law, let $N$ processors are used and $P$ is the percentage of execution time of the parallelized part of the program, the performance improvement by parallel processing is known to be

$S(N) = \frac{1}{(1-P) + \frac{P}{N}}$

参照: 並列処理で、どういうときにパフォーマンスがあがるのか(理論) : 並行処理、並列処理のあれこれ - Qiita (opens new window)

In brief, The ratio of parallelization should be large, and it should be kept in mind that there is a limit to the speedup of processing by parallelization.

However, as shown above(and references), the Julia program, in which most of the processing is parallelized, can be expected to be 1/20th as fast, so parallelization can be very effective. In other languages, implement the parallelization process is very difficult and requires a lot of skill, but in Julia it is very easy to implement as shown above.

# Reference

並行処理、並列処理のあれこれ - Qiita (opens new window)

Julia プログラミングクックブック ―言語仕様からデータ分析、機械学習、数値計算まで (opens new window)

The-@threads-Macro Multi-Threading · The Julia Language (opens new window)