by

piotr janiuk

golem: Three sided market

INTRODUCTION

challenges of decentralized setting

Game-theoretical standpoint for all actors

01

Trust in the network

02

INTRODUCTION

Strict verification requirement

03

Nondeterministic computations

04

https://truebit.io - can be used in case of deterministic computations

Monte Carlo methods

why rendering?

source: https://www.behance.net/gallery/21615831/Testing-render-engines

INTRODUCTION

L_o(x, \vec{\omega}) = L_e(x, \vec{\omega}) + \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) L_i(x, \vec{\omega}') \cos\theta d\vec{\omega}'
Lo(x,ω)=Le(x,ω)+Ωρ(x,ω,ω)Li(x,ω)cosθdωL_o(x, \vec{\omega}) = L_e(x, \vec{\omega}) + \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) L_i(x, \vec{\omega}') \cos\theta d\vec{\omega}'
\cos\theta = (\vec{\omega}' \cdot \vec{n})
cosθ=(ωn)\cos\theta = (\vec{\omega}' \cdot \vec{n})

The domain of the problem is well known

CG is present in most media, and pleasing results are Achieved fairly quickly

trust and security

INTRODUCTION

Concent

01

Intel SGX technology

02

  • Is a service in the Golem network, which aims to improve the integrity and security of transactions
  • Technology which can be used to implement secure remote computation

a brief intro to golem tech

Golem Technology

and Nondeterminism

 

first poc

Minilight (Python)

GOLEM TECHNOLOGY

Focus on computations / computation-centric use case

Specific renderer hardcoded in provider

(POC 0.5 - Minilight)

01

first poc

pbrt (C++)

GOLEM TECHNOLOGY

Focus on computations / computation-centric use case

  • Trusted computing nodes
  • Lack of transaction framework

  • No payments

Rendering as a separate task

02

Basic protocol

03

  • Rendering task is abstracted out and decoupled from the Golem client code. The first implementation used PBRT renderer as a rendering backend

first poc

First POC

01

Intranet computations

02

GOLEM TECHNOLOGY

TASK POV

 

Execution environment:

  • Multiple computational backends
  • Multiple operating systems
  • Multiple hardware configurations 

01

High level task definition

03

  • Timeouts (global, local)
  • Task splitting configuration

Subtask results collector

Task configuration:

02

04

GOLEM TECHNOLOGY

computation environment

GOLEM TECHNOLOGY

Physical infrastructure: Heterogenous p2p network

01

Logical components: Requestors, providers, (optional) concents

02

computation environment

GOLEM TECHNOLOGY

Task sending and resource distribution

03

Verification

04

security considerations

GOLEM TECHNOLOGY

Provider security

  • Isolating host from possibly malicious software
  • Accepting only valid tasks

01

Requestor security

02

  • Trust (i.e. reputation)
  • Results verification

  • Keep track of providers and notion of trust

Infrastructure security

03

  • no unauthorized internet access (e.g. botnet protection)

Infrastructure and actors

GOLEM TECHNOLOGY

New topology and requirements – P2P

01

Distribution of execution environments

02

Notion of trust – reputation of the actors

03

New integrations – new use-cases

04

Actors in the network

05

Incentives and economy

GOLEM TECHNOLOGY

Game-theoretical conditions for all actors

01

Requestor incentives

02

Provider incentives

03

  • Notion of trust in the network (local and global schemes)

  • Known verification requirements and algorithms

  • Open market

  • Wants to compute cheap, fast and get a high-quality result

  • Wants to compute fast and make the highest possible profit

payments inside the network

GOLEM TECHNOLOGY

Reward scheme:

01

  • Every integration can use any custom reward scheme, as long as it is compliant with the transaction framework

  • Key parties are:

    • task creators (Requestors): how they pay for computations and software

    • Providers: how they are rewarded (eg. per hour? per task?)

    • Developers: payment for software (free? donation? per node? per use? per CPU hour

  • Ethereum network is used to transfer value and information between the peers

a brief intro to golem tech

Golem Technology

and Nondeterminism

 

first use-case (mc photorealistic rendering)

Sources of nondeterminism

NONDETERMINISM

 

Random number generators

01

02

 

Floating point encoding standards on different platforms:

  • CPUs
  • GPUs

Sources of nondeterminism

NONDETERMINISM

 

Random number generators

01

02

 

Float encoding standards

on different platforms:

 

Multithreading:

  • Regular CPU multithreading

03

  • CPUs
  • GPUs

Sources of nondeterminism

NONDETERMINISM

 

Random number generators

01

02

 

Float encoding standards

on different platforms:

 

Multithreading:

  • Regular CPU multithreading

  • SIMT or SMT in the case of GPUs

03

  • CPUs
  • GPUs

cuda architecture

Sources of nondeterminism

NONDETERMINISM

 

Random number generators

01

02

 

Float encoding standards

on different platforms:

 

Multithreading:

  • Regular CPU multithreading

  • SIMT or SMT in the case of GPUs

03

  • CPUs
  • GPUs

 

Nondeterministic implementation

of MC methods

04

problems with nondeterminism

01

 

Rendering

NONDETERMINISM

  • Deterministic input

  • Nondeterministic computation

  • Nondeterministic results

problems with nondeterminism

 

Machine Learning

02

  • Deterministic or randomized input

  • Deterministic or nondeterministic computations

  • Nondeterministic results

NONDETERMINISM

problems with nondeterminism

 

Proof of Work

  • Deterministic input

  • Nondeterministic amount of work required

  • Deterministic and efficient verification

  • The exact amount of performed computations is hard to verify

03

NONDETERMINISM

Nondeterminism and consensus

01

NONDETERMINISM

 

Nondeterministic state

  • Nondeterministic results cannot be directly compared

  • Each node may see a slightly different result

  • A similarity measure is required

Nondeterminism and consensus

 

Golem's setting

  • Consensus between a very small  (usually two) number of nodes

  • Means of dealing with a dispute are implemented

02

NONDETERMINISM

monte carlo methods

Introduction

Basic probability

 

Probabilistic space

01

 

Probabilistic measure via density function

02

probabilistic space

– domain

probabilistic measure

probability density function

MONTE CARLO METHODS

\Omega
Ω\Omega
dP
dPdP
\int_{\Omega} dP(x) = 1
ΩdP(x)=1\int_{\Omega} dP(x) = 1
p(x) \ge 0
p(x)0p(x) \ge 0
dP(x) = p(x)dx
dP(x)=p(x)dxdP(x) = p(x)dx
\int_{\Omega} dP(x) = \int_{\Omega}p(x) dx = 1
ΩdP(x)=Ωp(x)dx=1\int_{\Omega} dP(x) = \int_{\Omega}p(x) dx = 1

Basic probability

 

Random variable

03

random variable

MONTE CARLO METHODS

P(x \in S) = \int_{S} dP(x) = \int_{S} p(x)dx
P(xS)=SdP(x)=Sp(x)dxP(x \in S) = \int_{S} dP(x) = \int_{S} p(x)dx
S \subseteq \Omega
SΩS \subseteq \Omega

A random variable, usually written X, is a variable whose possible values are numerical outcomes of a random phenomenon. There are two types of random variables, discrete and continuous.

– Valerie J. Easton and John H. McColl's Statistics Glossary v1.1

Basic PROBABILITY

 

Expected value

04

Continuous random variable expected value

definition

Linearity of random variables with regards to E

sum of i.i.D. random variable

MONTE CARLO METHODS

\mathrm{E}(x) = \int_{\Omega} x dP(x) = \int_{\Omega} x p(x) dx
E(x)=ΩxdP(x)=Ωxp(x)dx\mathrm{E}(x) = \int_{\Omega} x dP(x) = \int_{\Omega} x p(x) dx
\mathrm{Var}(X) = \mathrm{E}(X - \mathrm{E}X)^2
Var(X)=E(XEX)2\mathrm{Var}(X) = \mathrm{E}(X - \mathrm{E}X)^2
\mathrm{E}(aX + bY) = a\mathrm{E}X + b\mathrm{E}Y
E(aX+bY)=aEX+bEY\mathrm{E}(aX + bY) = a\mathrm{E}X + b\mathrm{E}Y
\mathrm{Var}(aX + bY) = a^2\mathrm{Var}X + b^2\mathrm{Var}Y
Var(aX+bY)=a2VarX+b2VarY\mathrm{Var}(aX + bY) = a^2\mathrm{Var}X + b^2\mathrm{Var}Y

 

Variance

05

Basic PROBABILITY

 

Vector of random variables

05

Sample mean

Expected value of sample mean

Variance of sample mean

Standard deviation

$$P(\mathrm{E}X = \lim_{N \rightarrow \infty} \frac{1}{N} \sum_{k=1}^{N}X_k) = 1$$

Strong law of large numbers

 

Strong law of large numbers

06

MONTE CARLO METHODS

\overline{X} = \frac{X_1 + X_2 + \ldots + X_n}{n}
X=X1+X2++Xnn\overline{X} = \frac{X_1 + X_2 + \ldots + X_n}{n}
\mathrm{E}\overline{X} = \frac{\mathrm{E}X_1 + \mathrm{E}X_2 + \ldots + \mathrm{E}X_n}{n} = \frac{n\mathrm{E}X}{n} = \mathrm{E}X
EX=EX1+EX2++EXnn=nEXn=EX\mathrm{E}\overline{X} = \frac{\mathrm{E}X_1 + \mathrm{E}X_2 + \ldots + \mathrm{E}X_n}{n} = \frac{n\mathrm{E}X}{n} = \mathrm{E}X
\mathrm{Var}(\overline{X}) = \frac{\mathrm{Var}(X_1) + \mathrm{Var}(X_2) + \ldots + \mathrm{Var}(X_n)}{n^2} = \frac{n\mathrm{Var}(X)}{n^2} = \frac{\mathrm{Var}(X)}{n}
Var(X)=Var(X1)+Var(X2)++Var(Xn)n2=nVar(X)n2=Var(X)n \mathrm{Var}(\overline{X}) = \frac{\mathrm{Var}(X_1) + \mathrm{Var}(X_2) + \ldots + \mathrm{Var}(X_n)}{n^2} = \frac{n\mathrm{Var}(X)}{n^2} = \frac{\mathrm{Var}(X)}{n}
\sigma \propto \frac{1}{\sqrt{N}}
σ1N\sigma \propto \frac{1}{\sqrt{N}}

monte carlo methods

Monte Carlo methods are a broad class of computational algorithms that rely on repeated random sampling to obtain numerical results.

Can be used to solve any problem having a probabilistic interpretation. 

By the law of large numbers, integrals described by the expected value of some random variable can be approximated by taking the empirical mean of independent samples of the variable.

– wikipedia

MONTE CARLO METHODS

approximation of     using MC methods

$$\pi$$

monte carlo integration

 

Goal

  • Find the best possible approximation of the integral by sampling the integrated function at a finite set of points

01

Integral

MONTE CARLO METHODS

$$\int_{S}f(x)dx$$

 

Expected Value approximation

02

Average with respect to some density p

$$\mathrm{E}(f(x)) = \int_{\Omega} f(x)p(x) dx \approx \frac{1}{N} \sum_{k=1}^{N}f(x_k) $$

Average estimation - uniform samples

$$P(\mathrm{E}X = \lim_{N \rightarrow \infty} \frac{1}{N} \sum_{k=1}^{N}X_k) = 1$$

Strong law of large numbers

$$\mathrm{E}(f(x)) = \int_{\Omega} f(x)dP(x) = \int_{\Omega} f(x)p(x) dx $$

monte carlo integration

 

Mean of a function

03

An average expressed as an expected value with uniform density function

Integral via mean

N uniformly distributed samples

$$\mu(\Omega) I = \mu(\Omega) \mathrm{E}(f(x)) \approx \mu(\Omega) \frac{1}{N} \sum_{k=1}^{N}f(x_k)$$

Integral Estimator

MONTE CARLO METHODS

p(x) = \frac{1}{\mu(\Omega)}
p(x)=1μ(Ω)p(x) = \frac{1}{\mu(\Omega)}
\mathrm{E}(f(x)) = \int_{\Omega} \frac{1}{\mu(\Omega)} f(x) dx = \frac{1}{\mu(\Omega)} \int_{\Omega} f(x) dx = I
E(f(x))=Ω1μ(Ω)f(x)dx=1μ(Ω)Ωf(x)dx=I\mathrm{E}(f(x)) = \int_{\Omega} \frac{1}{\mu(\Omega)} f(x) dx = \frac{1}{\mu(\Omega)} \int_{\Omega} f(x) dx = I
\mu({\Omega}) I = \mu({\Omega}) \frac{1}{\mu(\Omega)} \int_{\Omega}f(x)dx = \int_{\Omega}f(x)dx
μ(Ω)I=μ(Ω)1μ(Ω)Ωf(x)dx=Ωf(x)dx\mu({\Omega}) I = \mu({\Omega}) \frac{1}{\mu(\Omega)} \int_{\Omega}f(x)dx = \int_{\Omega}f(x)dx
x_k \sim U(\Omega)
xkU(Ω)x_k \sim U(\Omega)

 

Integral estimation using mean value

04

monte carlo integration

 

Integral expressed in terms of an arbitrary distribution

05

Average with respect to some density p

expected value of a slightly modified function

$$\int_{\Omega}f(x)dx = \int_{\Omega} \frac{f(x)}{p(x)}p(x) dx \approx \frac{1}{N} \sum_{k=1}^{N}\frac{f(x_k)}{p(x_k)}$$

Integral estimation - i.i.D. samples sampled from the distribution specified by P

where

independent identically distributred

a vector of N i.i.d. variables

MONTE CARLO METHODS

x_k \sim p
xkpx_k \sim p
x_k
xkx_k

$$\mathrm{E}(f(x)) = \int_{\Omega} f(x)dP(x) = \int_{\Omega} f(x)p(x) dx $$

$$\mathrm{E}(\frac{f(x)}{p(x)}) = \int_{\Omega} \frac{f(x)}{p(x)} p(x) dx = \int_{\Omega} f(x) dx$$

monte carlo integration

Standard deviation

 

Problem of diminishing return

06

MONTE CARLO METHODS

\sigma \propto \frac{1}{\sqrt{N}}
σ1N\sigma \propto \frac{1}{\sqrt{N}}

To halve the error the number of samples has to be quadrupled.

 

Variance reduction techniques

07

  • Choosing density wisely we can greatly reduce initial variance (Importance Sampling)

monte carlo integration

Standard deviation

 

Problem of diminishing return

06

MONTE CARLO METHODS

\sigma \propto \frac{1}{\sqrt{N}}
σ1N\sigma \propto \frac{1}{\sqrt{N}}

To halve the error the number of samples has to be quadrupled.

 

Variance reduction techniques

07

  • Domain partitioning (Stratified Sampling)

the rendering equation

rendering equation

In computer graphics, the rendering equation is an integral equation in which the equilibrium radiance leaving a point is given as the sum of emitted plus reflected radiance under a geometric optics approximation.

– wikipedia

RENDERING EQUATION

L_o(x, \vec{\omega}) = L_e(x, \vec{\omega}) + \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) L_i(x, \vec{\omega}') \cos\theta d\vec{\omega}'
Lo(x,ω)=Le(x,ω)+Ωρ(x,ω,ω)Li(x,ω)cosθdωL_o(x, \vec{\omega}) = L_e(x, \vec{\omega}) + \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) L_i(x, \vec{\omega}') \cos\theta d\vec{\omega}'
\cos\theta = (\vec{\omega}' \cdot \vec{n})
cosθ=(ωn)\cos\theta = (\vec{\omega}' \cdot \vec{n})

radiometric quantities

 

Energy

01

Energy

 

Flux

02

 

Radiance

03

Flux

radiance

$$\theta$$

RENDERING EQUATION

Q
QQ
\Phi = \dfrac{dQ}{dt}
Φ=dQdt\Phi = \dfrac{dQ}{dt}
L(x,\vec{\omega}) = \dfrac{d^2\Phi}{\cos\theta dA d\vec{\omega}}
L(x,ω)=d2ΦcosθdAdωL(x,\vec{\omega}) = \dfrac{d^2\Phi}{\cos\theta dA d\vec{\omega}}
\cos\theta = (\vec{\omega} \cdot \vec{n})
cosθ=(ωn)\cos\theta = (\vec{\omega} \cdot \vec{n})
\vec{n}
n\vec{n}
\vec{\omega}
ω\vec{\omega}
dA\cos\theta
dAcosθdA\cos\theta
  • Radiant flux per unit solid angle per unit projected area

  • Intuitively, radiance expresses how much power (flux) arrives at (or leaves from) a certain point on a surface, per unit solid angle, and per unit projected area

  • Time rate of flow of radiant energy

  • This quantity expresses how much total energy flows from/to/through a surface per unit time

  • Energy of a collection of photons

dA
dAdA

radiance propagation

 

Bidirectional Reflectance Distribution Function (BRDF)

  • A function that defines how light is reflected from a surface

01

 

Reflected radiance

02

BRDF

differential representation

integral form

RENDERING EQUATION

\vec{n}
n\vec{n}
\vec{\omega}
ω\vec{\omega}
\vec{\omega} '
ω\vec{\omega} '
\theta
θ\theta
\rho(x,\vec{\omega}', \vec{\omega}) = \dfrac{dL_r(x,\vec{\omega})}{L_i(x,\vec{\omega}') \cos\theta d\vec{\omega}'}
ρ(x,ω,ω)=dLr(x,ω)Li(x,ω)cosθdω\rho(x,\vec{\omega}', \vec{\omega}) = \dfrac{dL_r(x,\vec{\omega})}{L_i(x,\vec{\omega}') \cos\theta d\vec{\omega}'}
dL_r(x,\vec{\omega}) = \rho(x,\vec{\omega}' ,\vec{\omega}) L_i(x,\vec{\omega}') \cos\theta d\vec{\omega}'
dLr(x,ω)=ρ(x,ω,ω)Li(x,ω)cosθdωdL_r(x,\vec{\omega}) = \rho(x,\vec{\omega}' ,\vec{\omega}) L_i(x,\vec{\omega}') \cos\theta d\vec{\omega}'
L_r(x, \vec{\omega}) = \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) L_i(x, \vec{\omega}') \cos\theta d\vec{\omega}'
Lr(x,ω)=Ωρ(x,ω,ω)Li(x,ω)cosθdωL_r(x, \vec{\omega}) = \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) L_i(x, \vec{\omega}') \cos\theta d\vec{\omega}'
x
xx
\cos\theta = (\vec{\omega} ' \cdot \vec{n})
cosθ=(ωn)\cos\theta = (\vec{\omega} ' \cdot \vec{n})

$$x$$

dA
dAdA
L_r
LrL_r
d\vec{\omega}
dωd\vec{\omega}
L_i
LiL_i
L_i
LiL_i
L_i
LiL_i
L_i
LiL_i
L_i
LiL_i

rendering equation

 

Outgoing radiance

01

 

Rendering equation

02

Outgoing radiance

Rendering equation

RENDERING EQUATION

L_o(x, \vec{\omega}) = L_e(x, \vec{\omega}) + L_r(x, \vec{\omega})
Lo(x,ω)=Le(x,ω)+Lr(x,ω)L_o(x, \vec{\omega}) = L_e(x, \vec{\omega}) + L_r(x, \vec{\omega})
L_o(x, \vec{\omega}) = L_e(x, \vec{\omega}) + \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) L_i(x, \vec{\omega}') \cos\theta d\vec{\omega}'
Lo(x,ω)=Le(x,ω)+Ωρ(x,ω,ω)Li(x,ω)cosθdωL_o(x, \vec{\omega}) = L_e(x, \vec{\omega}) + \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) L_i(x, \vec{\omega}') \cos\theta d\vec{\omega}'
\cos\theta = (\vec{\omega}' \cdot \vec{n})
cosθ=(ωn)\cos\theta = (\vec{\omega}' \cdot \vec{n})

 

Operator formulation

03

integral form in operator notation

operator defined as

T = L_e + TL
T=Le+TLT = L_e + TL
< T g > (x, \vec{\omega}) = \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) g(x, \vec{\omega}') \cos\theta d\vec{\omega}'
<Tg>(x,ω)=Ωρ(x,ω,ω)g(x,ω)cosθdω< T g > (x, \vec{\omega}) = \int_{\Omega} \rho(x, \vec{\omega}', \vec{\omega}) g(x, \vec{\omega}') \cos\theta d\vec{\omega}'
\cos\theta = (\vec{\omega}' \cdot \vec{n})
cosθ=(ωn)\cos\theta = (\vec{\omega}' \cdot \vec{n})

rendering equation

 

Neumann Series

04

neumann series expansion

$$T = \sum_{k=0}^{\infty} T^kL_e$$

Compact notation

  • Renderers - black box engines

    • The rendering equation expressed via Neumann series expansion suggests that the final solution can be composed of independently computed results.

    • These partial results are computed by renderers, which can be treated as black box computation engines

RENDERING EQUATION

T = L_e + TL_e + T^2L_e + T^3L_e + \ldots
T=Le+TLe+T2Le+T3Le+T = L_e + TL_e + T^2L_e + T^3L_e + \ldots

rendering process

RENDERING EQUATION

Rendering or image synthesis is the automatic process of generating a photorealistic or non-photorealistic image from a 2D or 3D model (or models in what collectively could be called a scene file) by means of computer programs. Also, the results of displaying such a model can be called a render. A scene file contains objects in a strictly defined language or data structure; it would contain geometry, viewpoint, texture, lighting, and shading information as a description of the virtual scene. The data contained in the scene file is then passed to a rendering program to be processed and output to a digital image or raster graphics image file.

– wikipedia

(in this context) a computer program used to generate photorealistic images from a scene file (consisting of 3D models and description of materials and lights)

rendering

renderer

path tracing

RENDERING EQUATION

 

Path formulation

01

  • Markov Chain approach

  • Independent evaluation of each path

examples of path grouping

RENDERING EQUATION

  • Each tile is rendered independently, the final image is composed of tiles rendered with full quality

  • Multiple, independent full resolution images are rendered. Each one uses only a fraction of samples per pixel as compared to the final result. The final image is obtained by means of averaging these intermediate results

monte carlo methods

Verification challenges

Verification challenges

VERIFICATION CHALLENGES

 

Game-theoretical approach

01

 

Verification process outline

02

  • Each rational participant wants to maximize his/her benefit

  • 1st step - automatic verification

    1. Classifier can be easily replaced (it is a plugin architecture)

    2. The main classifier can consist of multiple classifiers and image comparison algorithms

  • 2nd step - manual verification and acceptance/rejection

hybrid approach

Verification challenges

VERIFICATION CHALLENGES

 

Automatic verification

03

  • Outlier removal method

  • Verification based on redundant computations (probabilistic)

outlier removal method

verification based on redundant computations

Verification challenges

VERIFICATION CHALLENGES

 

Manual verification

04

  • The final result may be accepted or rejected manually

manual Verification

consequences of decentralization

consequences of decentralization

Requestor prepares scenes and sends them together with the resources to a render farm

01

Render farm splits the task and distributes computations between nodes

02

Nodes compute partial results which are composed into final images/animations

03

The results are sent back to the requestor

04

default approach

DECENTRALIZATION

consequences of decentralization

decentralized setting

 

Requestor wants a valid result for the lowest price possible in a specified timeframe:

  • Must carefully set the number of samples for a valid result to avoid unnecessary computation

  • Won't pay for invalid results

  • Requires fair protocol in case of invalid results

01

DECENTRALIZATION

consequences of decentralization

decentralized setting

 

Provider wants to earn as much as possible for the least possible resources provided:

  • Can try to generate the image of lesser quality and use fewer samples than the requestor requested

  • May not be paid for valid results and needs to have a way of recovering the payment

  • Protocol must be fair

02

DECENTRALIZATION

verification in The case of monte carlo rendering

verification - monte carlo rendering example

VERIFICATION - MC

 

Blender (Cycles renderer) use-case

01

  • Bit-by-bit verification is not viable

  • Each tile has to be either marked as valid or invalid by automatic verification algorithm

verification - monte carlo rendering example

VERIFICATION - MC

 

Example of classifier implementation

  • Local verification

    • Sample rectangles are rendered with the same parameters as the main tile

    • Each sample must match the corresponding rectangle in the tile

    • Total area of sample rectangles is only a small fraction of the tile area

02

automatic Verification

verification - monte carlo rendering example

VERIFICATION - MC

 

Example of classifier implementation

 

  • Redundancy

    • Rendering each tile multiple times and comparing

    • Rendering redundantly but not necessarily duplicating all pixels

    • Multiple schemes are possible, which makes it harder to cheat

03

verification - monte carlo rendering example

VERIFICATION - MC

 

It is probabilistic

  • Tunable redundancy
  • Tunable verification thresholds

  • Tunable trust requirements (e.g. when SGX nodes are present)

01

summary

 

Classifier only assists

02

 

It is adjustable

03

image comparison and verification

image comparison and verification

COMPARISON AND VERIFICATION

 

Formal problem statement

01

Verification prerequisites

p = q
p=qp = q
b \in B
bBb \in B
  • For the same input parameters and renderer run in exactly the same conditions we have

Approach 1 - estimate the probability

provided that we know

P(q = p)
P(q=p)P(q = p)
p, b, R
p,b,Rp, b, R

input bitmap

R \in \Re
RR \in \Re
q \in P
qPq \in P
p \in P
pPp \in P

renderer

unknown parameters used to render

parameters specified by the requestor

b
bb
R(q) = R(p) = b
R(q)=R(p)=bR(q) = R(p) = b

APPROACH 2 - ESTIMATE THE PROBABILITY (preferred solution)

provided that we know

P(R(p) = b)
P(R(p)=b)P(R(p) = b)
p, b, R
p,b,Rp, b, R
R(p)
R(p)R(p)

but we do not want to recompute the whole

 

Estimation

02

image comparison and verification

COMPARISON AND VERIFICATION

 

Example approach

02

transform

  • Two transforms

    • ​​Wavelet transform

    • Edge enhancing transform

  • Two measures

    • SSIM (structural similarity)

    • MSE (mean squared error)

Similarity measure

 

Similarity of images

01

T_j \colon B \rightarrow B
Tj:BBT_j \colon B \rightarrow B
M_i \colon B \times B \rightarrow \mathbb{R}
Mi:B×BRM_i \colon B \times B \rightarrow \mathbb{R}
  • a decent algorithm is required to help the classifier decide if two bitmaps are similar enough to be treated as the result of the same computation

image comparison and verification

COMPARISON AND VERIFICATION

Picture w/ no transform

 

Importance of the transforms

03

Comparison of two bitmaps

  • Picture w/ no transform vs Picture w/ edge enhancing transform (one SPP difference in images)

m_{ji} = M_i(T_j(b_1), T_j(b_2))
mji=Mi(Tj(b1),Tj(b2))m_{ji} = M_i(T_j(b_1), T_j(b_2))

image comparison and verification

COMPARISON AND VERIFICATION

Picture w/ edge enhancing transform

 

Importance of the transforms

03

Comparison of two bitmaps

  • Picture w/ no transform vs Picture w/ edge enhancing transform (one SPP difference in images)

m_{ji} = M_i(T_j(b_1), T_j(b_2))
mji=Mi(Tj(b1),Tj(b2))m_{ji} = M_i(T_j(b_1), T_j(b_2))

other use-cases

COMPARISON AND VERIFICATION

 

Machine Learning

01

 

  • Golem can be used to find the best set of hyperparameters for a given model

  • Hyperparameters describe network structure, eg. number of layers, number of nodes in each layer. They are set before training

  • Each provider is doing a full training of a neural network but for different values of hyperparameters

  • Requestors can use different algorithms for searching parameter space, eg. grid search, random search, bayesian optimization search

  • Each dot in the picture represents one set of parameters, ie. one network trained by Provider

other use-cases

COMPARISON AND VERIFICATION

 

Machine Learning - Verification

02

  • Neural network training is a sequential algorithm

  • Feeding sets of batches to the network is one iteration step. After every step, the provider computes the hash of the network state and sends it to requestor

  • Requestor asks provider for randomly chosen states and verifies that transition to the next state was done correctly

  • Each rectangle represents one network state. The Requestor checks the transition between randomly chosen states

other use-cases

COMPARISON AND VERIFICATION

 

Proof of Work

01

Nonces for PoW

S=\{ 0, 1, \ldots, 2^{n-1} \}
S={0,1,,2n1}S=\{ 0, 1, \ldots, 2^{n-1} \}
\sigma = \Big(
σ=(\sigma = \Big(
0,\;\;\;\;\: 1,\;\;\;\;\: 2,\;\;\;\;\:\, \ldots, 2^{n-1}
0,1,2,,2n10,\;\;\;\;\: 1,\;\;\;\;\: 2,\;\;\;\;\:\, \ldots, 2^{n-1}
\sigma(0), \sigma(1), \sigma(2), \ldots, \sigma(2^{n-1})
σ(0),σ(1),σ(2),,σ(2n1)\sigma(0), \sigma(1), \sigma(2), \ldots, \sigma(2^{n-1})
\Big)
)\Big)
\bigg\{
{\bigg\{
\bigg\{
{\bigg\{

$$S_1$$

$$S_K$$

$$\bigcup_{i=1}^{K}S_i = S$$

$$S_i \cap S_ j= \emptyset$$

$$i \neq j$$

for

$$\ldots$$

\mathrm{Res}_i = \{\{ k \in S_i | H(k \oplus \omega) \leq t \}, \{ n \in S_i | H(n \oplus \omega) \leq T \} \}
Resi={{kSiH(kω)t},{nSiH(nω)T}}\mathrm{Res}_i = \{\{ k \in S_i | H(k \oplus \omega) \leq t \}, \{ n \in S_i | H(n \oplus \omega) \leq T \} \}

PoW targets

\omega
ω\omega
H
HH
T
TT
t
tt

PoW input string

PoW hash function

PoW target

PoW intermediate target (used during verification),

T < t
T<tT < t

concent

what is concent?

  • Providers should be paid for calculations they do

  • Requestors must get proper results (honest calculations)

  • If something goes wrong, an optional third party may be utilized to solve the dispute

what is concent?

  • It is a web service (which ultimately may be distributed)

  • It is a special type of node in the Golem Network

  • Its purpose is to improve the fairness of transactions

  • It is passive

  • It is optional

CONCENT

what concent does

 

Enforcing communication:

  • After a Requestor gives a task to a Provider and the Provider starts to compute, the communication must follow the protocol. If one party goes offline or rejects accepting proper messages, Concent can enforce the right communication, or ensure that the malicious party takes responsibility for the broken communication

01

CONCENT

what concent does

 

Additional verification:

  • Verification is performed by Requestors. If a Requestor rejects the correct result, the Provider calls Concent which performs additional verification. It is similar to the one performed by the Requestor but stronger. If the result turns out to be correct then Concent forces the Requestor to pay

02

CONCENT

what concent does

 

Enforcing payment:

  • If a Requestor does not pay on time for accepted results, then Concent will charge payments from the Requestor's deposit

03

CONCENT

how concent works

Requestor sends a task to a Provider

01

Provider calculates the task and sends the result to the Requestor

02

Requestor verifies and accepts the result

03

If the result is correct, the Requestor pays, if not they do not pay for it

04

communication in golem

CONCENT

Concent's role in solving conflicts

All Concent use-cases are meant to overcome flaws like attacks/frauds

01

Use-cases can be initialized by one or two parties

02

Use-case starts when a party does not receive a response, is not paid on time, or if their results are wrongly rejected

03

To use Concent, parties must agree to it at the beginning by putting deposits in GNT which are transferred to a dedicated Ethereum contract

04

Concent has special permissions on deposits

05

Concent is not a free service

06

CONCENT

why concent?

can fairness in transactions between requestors and providers be ensured?

 

Fairness means that Requestors do not pay for incorrect results and must always pay for correct results

01

 

Main options for ensuring fairness in transactions:

  • Use trusted third-party

  • Use blockchain and/or consensus in network

02

 

Verification is critical for defining Concent and non-Concent solutions

03

CONCENT

is concent reliable?

 

A Priori assumption: Providers and Requestors trust Concent

01

 

Under normal conditions Concent is called very rarely

02

 

Concent as a service is a single-point-of-failure and may be vulnerable to DDoS attacks

03

 

The principle is that Golem Network can calculate tasks even if Concent is down

04

 

Concent service will be more decentralized in the future

05

 

It suffices for Concent to be up 99% of time

06

CONCENT

can concent be decentralized?

 

In the future we plan to allow others to run valid Concent services (i.e. software developers may want to run Concent for their own applications)

01

 

Technologies like SGX provide trustworthy and non-controlled calculations that are considered decentralized and i.e. can perform additional verification

02

CONCENT

SGX TECHnology

sgx technology

SGX TECHNOLOGY

SGX is a set of CPU instructions and hardware platform enhancements that enable apps to create private areas within which code and associated data can be stored safe from compromise during execution. If used correctly, this protection can prevent compromise due to attacks from privileged software and many hardware attacks

– intel

sgx technology

 

Verifiable computation:

  • Prevents BIOS/OS/VMM/SMM/drivers attacks

  • Prevents bus snooping and memory tampering

  • Provides hardware measurement mechanism

  • Provides hardware attestation (local and remote)

  • Provides hardware sealing (to the enclave and to the author)

02

 

Example usages:

  • User authentication

  • Sensitive data processing

  • Verifiable computation

01

SGX TECHNOLOGY

– verifiable computation is PARTICULARLY important from golem's perspective

sgx technology

 

Enclaves:

  • Similar in spirit to a DLL, SO or Dynamic Library, plus some additional configuration data

  • Limited to computations only (no direct IO or OS calls allowed)

  • IO or OS calls available by means of OCALLs mechanism (configured statically)

  • Enclave

    • Run and hosted in an untrusted environment

    • Once instantiated it becomes a trusted part of the app (As there may appear attacks on SGX which cannot be foreseen today)

03

SGX TECHNOLOGY

TCS – THREAD CONTROL STRUCTURE

lifetime of an enclave

 

Enclave creation:

  • Binary payload (similar to a dll) which is run inside the enclave

  • Configuration data:

    • Stack size

    • Heap size

    • Threads (via Thread Control Structures)

    • Enclave author’s public key

    • Software version

    • Product id

01

 

Enclave Launch:

  • Currently launched by Launch Enclave provided by Intel

02

 

Enclave Usage:

  • Communication via ECALLs and OCALLs

03

intel sgx application

SGX TECHNOLOGY

Actors/Parties

 

ISV

01

 

Application

02

 

Application SGX Enclave

03

 

Intel IAS

04

SGX TECHNOLOGY

  • Enclave developer; assists in attestation; for now must be registered to Intel

  • Launches enclave and provides access to it; untrusted

  • Provides measurement and mrsigner for attestation and sealing; trustworthy

  • Intel Attestation Service - used to verify EPID during remote attestation

security related activities

 

Measurement

01


Attestation

02

 

Sealing

03

  • mrenclave – hash of vanilla enclave state including its security properties – used to prove that the enclave was instantiated correctly

  • Local – between enclaves on a single machine

  • Remote – to prove the enclave validity to a remote user/client

  • Used to securely store persistent data

  • Seal to current enclave – only the enclave with the matching measurement can unseal the data

  • To the author – used to unseal data in enclaves developed by the same author

local attestation

remote attestation

SGX TECHNOLOGY

cons of SGX TECHNOLOGY

  • Intel presence is required to attest quotes and partially to launch enclaves

    • Launching an enclave with all SGX features enabled requires Launch Enclave provided by Intel

    • Currently the Intel Attestation Service (IAS) is necessary for the remote attestation to work

  • ISV requires Intel approval

  • Remote attestation procedure does not require IAS by design and a perfectly valid remote attestation process without IAS can be implemented (once IAS is liberated)

SGX TECHNOLOGY

SGX and golem

SGX and golem

 

How SGX can be used to the advantage of Golem

01

  • Software Developer is ISV and currently can serve as attestation service - and connect to Intel to assure correctness of the enclave in question

    • Once a proof for an enclave is created by IAS, the proof can be subsequently verified offline

    • The protocol used to communicate with IAS is not critical from a security standpoint

  • Requestor and Concent are "hosting" Developers code

SGX AND GOLEM

SGX and golem

 

How SGX nodes can be added to Golem

03

  • Additional Concent use cases

    • Even though SGX nodes are present and arbitrage is not as necessary as it was, additional logic related to licensing models may still be required and should be implemented in a decentralized and secure manner

SGX AND GOLEM

  • When (if) SGX is liberated, then only ISV is required to have a high reputation (from requestors' point of view)

  • ISV has skin in the game to provide a valid enclave, so they should be trusted with attesting enclaves

 

SGX in Golem - possible future

02

SGX and golem

 

How SGX nodes can be added to Golem

03

 

  • The most resource consuming aspect of Concent is additional verification

    • Concent can delegate verification to SGX nodes increasing scalability and keeping the same level of trustworthiness. It may even happen that SGX nodes computations are considered as more trustful. This makes the most sense if SGX nodes compute noticeably slower than regular nodes

    • In such a setting a regular node may not be incentivized to call SGX nodes directly, but there would be a special case where an SGX node would be able to provably repeat computation and compare results with questionable ones

  • Other solutions

    • May encompass sharing logic between one group of nodes and the state and history on other nodes

SGX AND GOLEM

SGX and golem

 

Computations and Verification with SGX

04

  • We can run the whole computation inside an SGX enclave (if possible) and if the result is returned - it is the result of a valid computation and no verification is needed

    • The aforementioned approach may result in less security guarantees than the default SGX enclaves, but should still be much more secure than computations carried out the regular way

SGX AND GOLEM

SGX and golem

 

Computations and Verification with SGX

04

  • We can run the whole computation inside an SGX enclave (if possible) and if the result is returned - it is the result of a valid computation and no verification is needed
    • The aforementioned approach may result in less security guarantees than the default SGX enclaves, but should still be much more secure than computations carried out the regular way
    • SGX enclaves are restricted to computations only, but more general platforms allowing arbitrary computations inside the enclave are being researched and developed. We closely cooperate with Invisible Things Labs, the team/company solving problems in this exact area

SGX AND GOLEM

Graphene: example approach for the universal solution

SGX and golem

 

How SGX can help with verification

05

 

  • We can use SGX nodes and redundancy to help verify (with tunable probability) the results from regular nodes

    • Just as regular redundancy, but we know that results from SGX nodes are significantly more trustworthy and can be used as reference data

  • Additional verification logic can be implemented and run in an enclave whereas the main computation is performed on the regular host machine, but with intermediate results being sent to the Enclave

  • Performance

    • SGX supports EPC paging (enclave's private memory protected from the host and external, physical attacks), but it may be harmful to the performance

    • Depending on the performance impact of SGX, an appropriate fraction of calculations should be delegated to SGX nodes, the less impact, the more subtasks can be sent to SGX nodes (possibly for higher price)

SGX AND GOLEM

SGX and golem

 

Summary

01

 

  • In a perfect world SGX or similar technology:

    • Should provide mechanism for verifying that declared computations indeed took place

    • Should assure privacy of sensitive data

    • Should not impose large computational overhead

    • Requestors shouldn’t have to verify the results locally at all

  • SGX or similar technology can be used to help decentralize server solutions in a secure manner

  • The Golem network may consist of regular nodes, nodes with SGX and Concents using SGX

    • Non-SGX nodes are first class citizens but may require more mechanisms to assure valid results produced by them (e.g. local verification, redundant computations, reputation)

  • Such technology seems to be a perfect match for nondeterministic computations (as defined in previous sections)

SGX AND GOLEM

summary

summary

  • There is much more to Golem than verification of broadly defined nondeterministic tasks, but this seems to be one of the most prominent challenges in a decentralized and trustless setting
  • A detailed introduction to Monte Carlo rendering was presented to give you a better understanding of the problems one has to tackle when fully a decentralized infrastructure is to be prepared. As most of the issues are well known, applying them in a trustless decentralized setting requires additional thoughts in order to balance: quality, security, pricing, and efficiency
  • Our approach to this challenge was presented and we shared a few ideas and directions we’re already exploring in regard to improving general network security without sacrificing too much efficiency
  • A few approaches to verification of rendering tasks were shown along with the introduction of Concents, optional infrastructural component purposed with securing the network and allowing software developers to implement custom licensing logic
  • Then Intel SGX technology was shown as a potentially good fit with regard to both nondeterministic verification and decentralization of Concent

SUMMARY

Piotr janiuk

CTO, co-founder

 

Golem’s speaker at Devcon-0 and Devcon-1. Previously a designer and lead developer of Black Vision, a real-time rendering engine for TV broadcasting. Implemented the fastest (at the time) software jpeg2000 codec for DCP. Also took part in a few side projects related to p2p networks.

Interested in computer graphics, digital signal processing, cryptography, compilers, virtualization, blockchain tech, parallel computing, distributed computing, trustless computing and optimization techniques.

 

viggith@golem.network

https://www.linkedin.com/in/viggith/

 

thank you

www.golem.network

For the help with this presentation thanks to:

Substantial knowledge: Aleksandra Skrzypczak, Łukasz Gleń

Design: Jacek Muszyński, Urszula Trzaskowska (Cobaltblue)

Edit: Joanna Janiuk, Dan Horne

Golem Deep Dive

By The Golem Project

Golem Deep Dive

  • 3,916
Loading comments...

More from The Golem Project