Commit 4c06fc5b authored by Anastasios Kalogeropoulos's avatar Anastasios Kalogeropoulos
Browse files

Project added

parents
# NEC
Our blockchain practical repository for the NEC challenge
## Prerequisites
To run the network please make sure the following have been installed:
- hyperledger fabric binaries https://hyperledger-fabric.readthedocs.io/en/release-2.0/install.html
- Go and the GOPATH variable
- npm and node
- python3 and pip
- Packages needed: `flask`,`requests`,`flask_cors`,`cryptography`
- pm2 via (`sudo npm install pm2 -g`)
## Automatic Network Setup
Steps to run:
```
bash setup_network.sh
bash start_network.sh
bash stop_network.sh
```
More details:
To setup the network please run `bash setup_network.sh`. The script will:
- generate all crypto-material
- setup all apis and clients
The network can be started via `bash start_network.sh`, which will start the hyperledger network and all apis and clients.
The network can be stopped via `bash stop_network.sh`, which will stop the network and shutdown all containers.
NOTE: sometimes HF Fabric gives various non-deterministic errors, please restart the network in that case.
## How to Run Demo
The full demo is here: https://www.youtube.com/watch?v=lyzwGfcaPJI
A more raw demo can be seen here: https://www.youtube.com/watch?v=10BqdfQ9dQU
Note, for the aggregator to perform the aggregation you need to make a HTTP POST to `http://127.0.0.1:11900/putAggDataOnBlockchain/`, with a JSON data load containing for example: `{"query_id": "q2"}`
All other operations are done via the UIs.
## Manual Network Setup
Run the commands in `start_network.sh` up until step 7 from the `/fabric` folder. Then go to the `/clients` folder, and start in 9 new terminals. For each terminal, start the respective Python clients or React acts (see `start_network.sh` for their locations). To start the apps you do not need any parameters other than `python3`/`npm start`. Note: first start the Python clients (Managing organization first), then the react apps. See the `/clients` folder for some examples.
To look into various blockchain aspects, look into the `/fabric` folder.
## Project structure
All network code is located in `/fabric/`. See readme there for additional info.
All client code is located in `/clients`. See readme there for additional info.
Various documentation is located in `docs_fabric_api` that was used previously for smart contract testing.
## Smart Contracts
- Located in basic network `/fabric/artifacts/src/`
- JS for smart contracts
- Coin smart contract, Query smart contract & Aggregated Answer smart contract
## API to HF network
- Readme examples in `/fabric_api/`
- Node server for API
- Located in `/fabric/api-2.0`
- `HTTP GET` applies a query on the blockchain
- `HTTP POST` applied an invoke on the blockchain
- Example to run and interact with QueryContract in `/docs_fabric_api/query`
## Logs and Errors
The logs of the python clients can be found in the `nohup.out` files located next to the python files. For node apps, all logs can be viewed via `pm2 logs`, e.g. `pm2 logs API`.
To view the logs of the HF containers, first find using `docker ps -a` the interesting container, and then use `docker logs <id>`.
# Clients
Here we have various clients written in python flask, which interact with the respective user and Fabric Network.
The user interfaces are written in React.
First start the python clients, e.g. `python3 ./managing_org/mo_client.py`. Note: always start the MO first.
Next start the react GUIs from the folder, e.g.:
```
cd managing_org/mo-webui
npm start
```
venv
nohup.out
\ No newline at end of file
import flask
from flask import request, jsonify
import requests
from multiprocessing.dummy import Pool
from cryptography.fernet import Fernet
import json
from flask_cors import CORS, cross_origin
# Flask config
app = flask.Flask(__name__)
app.config["DEBUG"] = False
local_port = 11700
cors = CORS(app)
# Other client URL config
addr_oo = "http://localhost:11500"
addr_mo_server = "http://localhost:11600"
addr_mo_user_api = "http://localhost:11620"
addr_dc = "http://localhost:11700"
addr_user = "http://localhost:11800"
addr_agg = "http://localhost:11900"
# HF connection config
addr_hf_api = "http://localhost:4000"
org_id = str(2) # TODO: needs to be changed to ??????????????????????
# Helper code
pool = Pool(10)
# logging
@app.before_request
def store_requests():
url = request.url
if "getRequestsHistory" not in url and "getQueryData" not in url and "getAllQueries" not in url:
logic.requests_log.append(url)
"""
Function to call a HTTP request async and only printing the result.
to_get: Bool, True = GET, False = POST
url: string URL to call
data: list/dict for POST data
headers: list/dict of headers
params: list/dict for GET params
"""
def fire_and_forget(to_get, url, data=[], headers=[], params=[]):
# Example: fire_and_forget(True, "https://www.google.com/")
def on_success(r):
if r.status_code == 200:
print(f'Call succeed: {r}')
else:
print(f'Call failed: {r}')
def on_error(ex: Exception):
print(f'Requests failed: {ex}')
if to_get:
pool.apply_async(requests.get, (url,), {'params': params, 'headers': headers},
callback=on_success, error_callback=on_error)
else:
pool.apply_async(requests.post, (url,), {'data': data, 'headers': headers},
callback=on_success, error_callback=on_error)
"""
Returns a HF API token needed for communication.
return: string of token
"""
def register_hf_api_token():
# NOTE: not tested fully
url_users = addr_hf_api + "/users"
org_str = "Org" + str(org_id)
username = "default_user_agg"
json_users = {"username": username, "orgName": org_str}
r = requests.post(url_users, data=json_users)
token = r.json()['token']
r.close()
return token
"""
Function to invoke (= HTTP POST) something from a chaincode via the HF API.
token: token which is returned by register_hf_api_token
chaincode_name: name of chaincode
function: function name
args: list of string args
returns: json of response
"""
def hf_invoke(token, chaincode_name, function, args):
# NOTE: not tested fully
url_post = addr_hf_api + "/channels/mychannel/chaincodes/" + chaincode_name
body = {"fcn": function,
"peers": "peer0.org" + org_id + ".example.com",
"chaincodeName": chaincode_name,
"channelName": "myChannel",
"args": args}
headers = {"Authorization": "Bearer " + token}
r = requests.post(url_post, data=body, headers=headers)
return r.json()
"""
Function to get something from a chaincode via the HF API.
token: token which is returned by register_hf_api_token
chaincode_name: name of chaincode
function: function name
args: list of string args
returns: json of response
"""
def hf_get(token, chaincode_name, function, args):
# NOTE: not tested fully
headers = {"Authorization": "Bearer " + token}
params = {"args": args,
"peer": "peer0.org" + org_id + ".example.com",
"fcn": function}
url_req = addr_hf_api + "/channels/mychannel/chaincodes/" + chaincode_name
r = requests.get(url_req, headers=headers, params=params)
return r.json()
"""
Helper function to invoke async operations i.e. you call the function but do not wait for the response.
function: reference to function to call
args: tuple of args to pass to function
"""
def invoke_async_function(function, args):
pool.apply_async(function, args)
# Logic class
class DCClientLogic(object):
def __init__(self):
self.dc_id = "dc1" # id - id of DC
self.hf_api_token = None # str - String of HF API token
self.wallet_id = "w134" # TODO: change id - wallet id of dc
self.query_id = None # int - Query's id
self.priv_key = None # Private keys of aggregated data
self.aggregated_unencrypted_data = None # Aggregated unencrypted data
self.requests_log = []
# TESTED, WORKS CORRECTLY
def createQuery(self, query_text, min_users, max_budget):
#create query and put it on HF, save the queryID as well. We need it later to check query's stage
hf_res = hf_invoke(self.hf_api_token, "query_contract", "createQuery", [query_text,
str(min_users),
str(max_budget),
str(self.wallet_id)])
self.query_id = hf_res['result']['message']
return self.query_id
# TESTED, WORKS CORRECTLY
def checkQuery(self, q_id):
#check the query's stage on the HF
query_stage = json.loads(hf_get(self.hf_api_token, "query_contract", "getQuery", [q_id])['result']['result'])['stage']
switcher = {
0: "FAILED",
1: "AWAITING_APPROVAL",
2: "APPROVED",
3: "CHECKING_USERS",
4: "SERVING_DATA",
5: "SERVED"
}
return switcher.get(query_stage, "Invalid query stage, ERROR!")
# TESTED, WORKS CORRECTLY
def createWallet(self):
#create wallet and send it to MO
hf_res = hf_invoke(self.hf_api_token, "coin_contract", "createWallet", [])
self.wallet_id = hf_res['result']['result']['id']
self.sendWalletToMO()
return self.wallet_id
# TESTED, WORKS CORRECTLY
def sendWalletToMO(self):
#send wallet to MO
try:
MO_endpoint = addr_mo_server + '/receiveDCWallet/' + str(self.dc_id) + '/' + str(self.wallet_id) + '/'
requests.post(MO_endpoint)
except Exception as e:
err = "Wallet not created, MO server probably not running or connection error"
print(err)
# NOT TESTED, NEEDS AGGREGATED ANSWER CONTRACT TO BE FINISHED
def getAggAnswerFromHF(self, q_id):
#get agg answer from HF, then decrypt it and save it on the logic class
hf_res = hf_get(self.hf_api_token, "agg_answer", "getAnswer", [q_id])
aggregated_encrypted_answer = json.loads(hf_res['result']['result'])['encr_answer_text']
cipher_suite = Fernet(self.priv_key)
self.aggregated_unencrypted_data = cipher_suite.decrypt(str.encode(aggregated_encrypted_answer)).decode('utf-8')
return self.aggregated_unencrypted_data
# Logic instance
logic = DCClientLogic()
# Endpoint management
# TESTED, WORKS CORRECTLY
@app.route('/createQuery/', methods=['POST'])
def createQuery():
#just call the logic's createQuery function with the arguments passed to the endpoint
try:
body = request.get_json(force=True)
query_text = body['query_text']
min_users = body['min_users']
max_budget = body['max_budget']
query_id = logic.createQuery(query_text, min_users, max_budget)
print("Added query from: " + str(logic.wallet_id) + ", the created query has id: " + str(query_id))
return jsonify(query_id=query_id)
except Exception as e:
return jsonify(error=str(e))
# TESTED, WORKS CORRECTLY
@app.route('/checkQueryStage/<query_id>/', methods=['GET'])
def checkQueryStage(query_id):
#just call the logic's checkQuery function
query_stage = logic.checkQuery(query_id)
print("Query stage is " + query_stage)
return jsonify(stage=query_stage)
# TESTED, WORKS CORRECTLY
@app.route('/receiveAggAnswer/', methods=['POST'])
def receiveKeyAndAnsId():
#receive privateKey and answer id from the AggregatorClient; save them on the logic class
try:
body = request.get_json()
logic.query_id = body['query_id']
logic.priv_key = str.encode(body['key'])
print("Query Id received: " + str(logic.query_id) + ", Private key received: " + str(logic.priv_key))
return jsonify(success=True)
except Exception as e:
print(e)
return jsonify(error=str(e))
#TESTED, WORKS CORRECTLY
@app.route('/getAnswerFromHF/<query_id>/', methods=['GET'])
def getAggAnswerFromHF(query_id):
#just call the logic's getAggAnswerFromHF function and return the decrypted data
unencrypted_data = logic.getAggAnswerFromHF(query_id)
print("Aggregated unencrypted answer is: " + str(unencrypted_data))
return jsonify(data=unencrypted_data)
@app.route('/getRequestsHistory/', methods=['GET'])
def get_requests_history():
return jsonify({"requests":logic.requests_log})
@app.errorhandler(500)
def page_not_found(e):
return jsonify(error=str(e))
@app.errorhandler(404)
def page_not_found(e):
return jsonify(error=str(e))
@app.errorhandler(403)
def page_not_found(e):
return jsonify(error=str(e))
if __name__ == '__main__':
hf_token = register_hf_api_token()
logic.hf_api_token = hf_token
logic.createWallet()
app.run(port=local_port)
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
# dependencies
/node_modules
/.pnp
.pnp.js
# testing
/coverage
# production
/build
# misc
.DS_Store
.env.local
.env.development.local
.env.test.local
.env.production.local
npm-debug.log*
yarn-debug.log*
yarn-error.log*
This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
## Available Scripts
In the project directory, you can run:
### `npm start`
Runs the app in the development mode.<br />
Open [http://localhost:3000](http://localhost:3000) to view it in the browser.
The page will reload if you make edits.<br />
You will also see any lint errors in the console.
### `npm test`
Launches the test runner in the interactive watch mode.<br />
See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
### `npm run build`
Builds the app for production to the `build` folder.<br />
It correctly bundles React in production mode and optimizes the build for the best performance.
The build is minified and the filenames include the hashes.<br />
Your app is ready to be deployed!
See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.
### `npm run eject`
**Note: this is a one-way operation. Once you `eject`, you can’t go back!**
If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.
Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own.
You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it.
## Learn More
You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
To learn React, check out the [React documentation](https://reactjs.org/).
### Code Splitting
This section has moved here: https://facebook.github.io/create-react-app/docs/code-splitting
### Analyzing the Bundle Size
This section has moved here: https://facebook.github.io/create-react-app/docs/analyzing-the-bundle-size
### Making a Progressive Web App
This section has moved here: https://facebook.github.io/create-react-app/docs/making-a-progressive-web-app
### Advanced Configuration
This section has moved here: https://facebook.github.io/create-react-app/docs/advanced-configuration
### Deployment
This section has moved here: https://facebook.github.io/create-react-app/docs/deployment
### `npm run build` fails to minify
This section has moved here: https://facebook.github.io/create-react-app/docs/troubleshooting#npm-run-build-fails-to-minify
This diff is collapsed.
{
"name": "mo-webui",
"version": "0.1.0",
"private": true,
"dependencies": {
"@material-ui/core": "^4.11.0",
"@material-ui/icons": "^4.9.1",
"@testing-library/jest-dom": "^4.2.4",
"@testing-library/react": "^9.5.0",
"@testing-library/user-event": "^7.2.1",
"axios": "^0.19.2",
"react": "^16.13.1",
"react-dom": "^16.13.1",
"react-scripts": "3.4.1"
},
"scripts": {
"start": "PORT=3001 react-scripts start",
"build": "react-scripts build",
"test": "react-scripts test",
"eject": "react-scripts eject"
},
"eslintConfig": {
"extends": "react-app"
},
"browserslist": {
"production": [
">0.2%",
"not dead",
"not op_mini all"
],
"development": [
"last 1 chrome version",
"last 1 firefox version",
"last 1 safari version"
]
}
}
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<link rel="icon" href="%PUBLIC_URL%/favicon.ico" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="theme-color" content="#000000" />
<meta
name="description"
content="Web site created using create-react-app"
/>
<link rel="apple-touch-icon" href="%PUBLIC_URL%/logo192.png" />
<!--
manifest.json provides metadata used when your web app is installed on a
user's mobile device or desktop. See https://developers.google.com/web/fundamentals/web-app-manifest/
-->
<link rel="manifest" href="%PUBLIC_URL%/manifest.json" />
<!--
Notice the use of %PUBLIC_URL% in the tags above.
It will be replaced with the URL of the `public` folder during the build.
Only files inside the `public` folder can be referenced from the HTML.
Unlike "/favicon.ico" or "favicon.ico", "%PUBLIC_URL%/favicon.ico" will
work correctly both with client-side routing and a non-root public URL.
Learn how to configure a non-root public URL by running `npm run build`.
-->
<title>React App</title>
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root"></div>
<!--
This HTML file is a template.
If you open it directly in the browser, you will see an empty page.
You can add webfonts, meta tags, or analytics to this file.
The build step will place the bundled scripts into the <body> tag.
To begin the development, run `npm start` or `yarn start`.
To create a production bundle, use `npm run build` or `yarn build`.
-->
</body>
</html>
{
"short_name": "React App",
"name": "Create React App Sample",
"icons": [
{
"src": "favicon.ico",
"sizes": "64x64 32x32 24x24 16x16",
"type": "image/x-icon"
},
{
"src": "logo192.png",
"type": "image/png",
"sizes": "192x192"
},
{
"src": "logo512.png",
"type": "image/png",
"sizes": "512x512"
}
],
"start_url": ".",
"display": "standalone",
"theme_color": "#000000",
"background_color": "#ffffff"
}
# https://www.robotstxt.org/robotstxt.html
User-agent: *
Disallow:
.App {
text-align: center;
}
.App-logo {
height: 40vmin;