Skip to content
Snippets Groups Projects
Unverified Commit b156b311 authored by dlorenc's avatar dlorenc Committed by GitHub
Browse files

Merge pull request #18 from lukehinds/cli-cleanup 

General clean up
parents 5b530723 3958a6da
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
README.md
+ 13
− 97
View file @ b156b311
# Rekor
### Note that this CLI application will undergo refactoring to point at the rekor-server instead of direct connections to the trillian backend (which is what it does right now).
# Rekor Command Line Interface
Early Development / Experimental use only.
Attestation and provenance of software, its generated artefacts and information on tools used to build said software, relies on an often disparate set of different approaches and data formats. The solutions that do exist, often rely on digests that are stored on insecure systems that are susceptible to tampering and can lead to various attacks such as swapping out of digests , replay attacks.
The goal of rekor would be to create a ledger service, and associated tooling for software maintainers to store metadata and digests of their software source code, artefacts and build process along with binary provenance. The ledger service will then act as a means for users to query said metadata and and assess the trust state / audit record of objects consumed within their own supply chain (for example dependencies).
Rekór - Greek for “Record”
Rekor's goals are to provide an immutable tamper resistant ledger of metadata generated within a software project or supply chain. Rekor would enable software maintainers and build systems to generate metadata containing signed digests to an immutable record. Other parties can then query said metadata to enable them to make informed decisions on trust and nonrepudiation of an object's lifecycle, based on signed metadata stored within a tamper proof binary (merkle) tree.
Rekor seeks to provide provenance and integrity of the software supply chain.
Provenance deals with systematically capturing metadata describing the relationships among all the elements such as source code, build tools / compiler, processing steps, contextual information and dependencies used. Software provenance can be used for many purposes, such as understanding how an artifact was collected, determining ownership and rights over an artifact for policy decisions, making judgements about information to determine whether to trust an external library, verifying whether the process and steps used to obtain an artifact are compliant with given requirements etc.
Integrity is a control mechanism that examines objects and checks if their integrity is intact and of a non tampered state. This is typically achieved using a cryptographically signed digest of the object (for example, code file, binary, configuration file). The signed digest and then be used to attest the trust status and provide surety that no unauthorised or malicious changes have been made.
It uses a trillian backend to store [in-toto](https://in-toto.io/) style metadata into an immutable merkle tree.
The rough idea is that a developer would include an in-toto style `.link` file along with a software
package release and then use rekor to make a transparency log entry with the same link file.
A receiver of the package would then use rekor to perform a `rekor get` command using the exact
same link file (that they would have received along with the released package)
If the link file is un-tampered, then they know the can trust the sha256 digests of the file, and the
developer's signature embedded within the in-toto link file.
If you had not already noticed, rekor is in very early development, so its not ready for production
use, however if you would like to contribute, then please do.
Its very simple at the moment, but plans are to work more with link files and other manifest structures
to allow automation of integrity checks and design how other link file content such as materials can
be stored and then queried in a useful manner.
The trillian components are:
* Rekor CLI
* Rekor trillian personality
* Trillian Log Server
* Trillian Log Signer
# Trillian, what's that?
Trillian provides the transparancy log. It allows population and query of
a distributable merkle tree.
Its sort of like blockchain, without the large electricity bills.
## Create Database
Trillian requires a database, we use MariaDB in this instance. Once this
is installed on your machine edit the `scripts/createdb.sh` with your
database root account credentials and run the script.
## Build Trillian
To run rekor you need to build trillian
```
go get github.com/google/trillian.git
go build ./cmd/trillian_log_server
go build ./cmd/trillian_log_signer
go build ./cmd/createtree/
```
### Start the tlog server
```
trillian_log_server -http_endpoint=localhost:8090 -rpc_endpoint=localhost:8091 --logtostderr ...
```
### Start the tlog signer
```
trillian_log_signer --logtostderr --force_master --http_endpoint=localhost:8190 -rpc_endpoint=localhost:8191 --batch_size=1000 --sequencer_guard_window=0 --sequencer_interval=200ms
```
Rekor's goals are to provide an immutable tamper resistant ledger of metadata generated within a software projects supply chain. Rekor will enable software maintainers and build systems to record signed metadata to an immutable record. Other parties can then query said metadata to enable them to make informed decisions on trust and nonrepudiation of an object's lifecycle, based on signed metadata stored within a tamper proof binary (merkle) tree.
## Create a tree (note the return value, you need this for the "tlog_id" flag)
The Rekor CLI requires a running instance of the [rekor-server](https://github.com/projectrekor/rekor-server).
```
./createtree --admin_server=localhost:8091 > logid
cat logid
2587331608088442751
```
The CLI will default to using a rekor server connection of `localhost:3000`, should you wish to use a different address:port, use the `--rekor_server` flag.
### Make an entry:
## Add an entry
```
rekor add --tlog_id=2587331608088442751 --linkfile tests/package.link
```
The `add` command sends a file to the transparency log, who then adds the file to the transparency log as a merkle leaf.
### Verify Inclusion Proof:
`rekor add --linkfile <your/linkfile.link>`
```
rekor inclusion --tlog_id=2587331608088442751 --linkfile tests/package.link
```
## Get Proof of Entry
### Query an entry:
`rekor get --linkfile <your/linkfile.link>`
```
rekor get --tlog_id=2587331608088442751 --linkfile tests/package.link
```
The `get` command performs an inclusion proof request to the transparency log. Atttributes such as the files merkle hash, signed tree root hash are used
to cryptographically verify proof of entry.
Should your file be returned in full, good news, it matches.
## Contributions and Issues
Should no return occur, then something is up (this of course will be handled
better in time).
Contributions are welcome, please fork and make a pull request. Likewise if you find an issue, please do raise it.
cmd/add.go
+ 4
− 3
View file @ b156b311
......@@ -64,10 +64,11 @@ import (
// addCmd represents the add command
var addCmd = &cobra.Command{
Use: "add",
Short: "Rekor CLI",
Long: `Rekor interacts with a transparency log
Short: "Rekor Add Command",
Long: `Add a linkfile to rekor
For more information, visit [domain]`,
The Add command will send a link file to rekor which will
then hash the file into the transparency log`,
Run: func(cmd *cobra.Command, args []string) {
log := log.Logger
......
cmd/get.go
+ 3
− 3
View file @ b156b311
......@@ -46,10 +46,10 @@ type getProofResponse struct {
// getCmd represents the get command
var getCmd = &cobra.Command{
Use: "get",
Short: "Rekor CLI",
Long: `Rekor interacts with a transparency log
Short: "Rekor get command",
Long: `Performs a proof verification that a file
For more information, visit [domain]`,
exists within the transparency log`,
Run: func(cmd *cobra.Command, args []string) {
log := log.Logger
rekorServer := viper.GetString("rekor_server")
......
cmd/inclusion.go deleted 100644 → 0
+ 0
− 77
View file @ 5b530723
/*
Copyright © 2020 NAME HERE <EMAIL ADDRESS>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package cmd
import (
"context"
"fmt"
"io/ioutil"
"os"
"time"
"github.com/google/trillian"
"github.com/projectrekor/rekor-cli/log"
"github.com/spf13/cobra"
"github.com/spf13/viper"
"google.golang.org/grpc"
)
// inclusionCmd represents the inclusion command
var inclusionCmd = &cobra.Command{
Use: "inclusion",
Short: "Rekor CLI",
Long: `Rekor interacts with a transparency log
For more information, visit [domain]`,
Run: func(cmd *cobra.Command, args []string) {
log := log.Logger
logRpcServer := viper.GetString("log_rpc_server")
tLogID := viper.GetInt64("tlog_id")
linkfile := viper.GetString("linkfile")
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
// Set up and test connection to rpc server
conn, err := grpc.DialContext(ctx, logRpcServer, grpc.WithInsecure())
if err != nil {
log.Fatal("Failed to connect to log server:", err)
}
defer conn.Close()
jsonFile, err := os.Open(linkfile)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
byteValue, _ := ioutil.ReadAll(jsonFile)
defer jsonFile.Close()
tLogClient := trillian.NewTrillianLogClient(conn)
server := serverInstance(tLogClient, tLogID)
resp := &Response{}
resp, err = server.getInclusion(byteValue, tLogID)
log.Infof("Server PUT Response: %s", resp)
},
}
func init() {
rootCmd.AddCommand(inclusionCmd)
}
cmd/log_server.go deleted 100644 → 0
+ 0
− 172
View file @ 5b530723
/*
Copyright © 2020 Luke Hinds <lhinds@redhat.com>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/
package cmd
import (
"context"
"fmt"
"time"
"github.com/projectrekor/rekor-cli/log"
"github.com/google/trillian"
"github.com/google/trillian/client"
"github.com/google/trillian/merkle"
"github.com/google/trillian/merkle/rfc6962"
"github.com/google/trillian/types"
"google.golang.org/grpc/codes"
)
type server struct {
lc *client.LogClient
client trillian.TrillianLogClient
logID int64
ctx context.Context
}
type Response struct {
status string
leafhash string
}
func serverInstance(client trillian.TrillianLogClient, tLogID int64) *server {
return &server{
client: client,
logID: tLogID,
}
}
func (s *server) root() (types.LogRootV1, error) {
rqst := &trillian.GetLatestSignedLogRootRequest{
LogId: s.logID,
}
resp, err := s.client.GetLatestSignedLogRoot(context.Background(), rqst)
if err != nil {
return types.LogRootV1{}, err
}
var root types.LogRootV1
if err := root.UnmarshalBinary(resp.SignedLogRoot.LogRoot); err != nil {
return types.LogRootV1{}, err
}
return root, nil
}
func (s *server) getInclusion(byteValue []byte, tLogID int64) (*Response, error) {
log := log.Logger
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
root, err := s.root()
if err != nil {
return &Response{}, err
}
log.Infof("Root hash: %x", root.RootHash)
hasher := rfc6962.DefaultHasher
leafHash := hasher.HashLeaf(byteValue)
resp, err := s.client.GetInclusionProofByHash(ctx,
&trillian.GetInclusionProofByHashRequest{
LogId: tLogID,
LeafHash: leafHash,
TreeSize: int64(root.TreeSize),
})
if err != nil {
log.Error(codes.Internal, "failed to get inclusion proof: %v", err)
return &Response{}, nil
}
if err != nil {
log.Info("two")
return &Response{}, err
}
if len(resp.Proof) < 1 {
log.Info("three")
return &Response{}, nil
}
v := merkle.NewLogVerifier(rfc6962.DefaultHasher)
for i, proof := range resp.Proof {
hashes := proof.GetHashes()
for j, hash := range hashes {
log.Infof("Proof[%d],hash[%d] == %x\n", i, j, hash)
}
if err := v.VerifyInclusionProof(proof.LeafIndex, int64(root.TreeSize), hashes, root.RootHash, leafHash); err != nil {
return &Response{}, err
}
}
return &Response{
status: "OK",
}, nil
}
func (s *server) addLeaf(byteValue []byte, tLogID int64) (*Response, error) {
log := log.Logger
leaf := &trillian.LogLeaf{
LeafValue: byteValue,
}
rqst := &trillian.QueueLeafRequest{
LogId: tLogID,
Leaf: leaf,
}
resp, err := s.client.QueueLeaf(context.Background(), rqst)
if err != nil {
fmt.Println(err)
}
c := codes.Code(resp.QueuedLeaf.GetStatus().GetCode())
if c != codes.OK && c != codes.AlreadyExists {
log.Error("Server Status: Bad status: %v", resp.QueuedLeaf.GetStatus())
}
if c == codes.OK {
log.Info("Server status: ok")
} else if c == codes.AlreadyExists {
log.Error("Data already Exists")
}
return &Response{
status: "OK",
}, nil
}
func (s *server) getLeaf(byteValue []byte, tlog_id int64) (*Response, error) {
log := log.Logger
hasher := rfc6962.DefaultHasher
leafHash := hasher.HashLeaf(byteValue)
rqst := &trillian.GetLeavesByHashRequest{
LogId: tlog_id,
LeafHash: [][]byte{leafHash},
}
resp, err := s.client.GetLeavesByHash(context.Background(), rqst)
if err != nil {
log.Fatal(err)
}
for i, logLeaf := range resp.GetLeaves() {
leafValue := logLeaf.GetLeafValue()
log.Infof("[server:get] %d: %s", i, leafValue)
}
return &Response{
status: "OK",
}, nil
}
cmd/root.go
+ 1
− 3
View file @ b156b311
......@@ -30,9 +30,7 @@ var cfgFile string
var rootCmd = &cobra.Command{
Use: "rekor",
Short: "Rekor CLI",
Long: `Rekor transparency log CLI
For more information, visit [domain]`,
Long: `Rekor command line interface tool`,
}
func Execute() {
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment