Documentation
¶
Index ¶
- Constants
- func FileProcessingPipeline(replayFile string, grpcAnonymizer *GRPCAnonymizer, ...) (bool, replay_data.CleanedReplay, persistent_data.ReplaySummary, string)
- func MultiprocessingChunkPipeline(listOfFiles []string, packageToZipBool bool, compressionMethod uint16, ...)
- func PipelineWrapper(fileChunks [][]string, packageToZipBool bool, compressionMethod uint16, ...)
- type GRPCAnonymizer
- type ReplayProcessingChannelContents
- type VerifyGameInfo
Constants ¶
View Source
const ( Ranked1v1 = 1 << iota Ranked2v2 Ranked3v3 Ranked4v4 RankedArchon Custom1v1 Custom2v2 Custom3v3 Custom4v4 CustomFFA )
Variables ¶
This section is empty.
Functions ¶
func FileProcessingPipeline ¶
func FileProcessingPipeline( replayFile string, grpcAnonymizer *GRPCAnonymizer, englishToForeignMapping map[string]string, cliFlags utils.CLIFlags, ) (bool, replay_data.CleanedReplay, persistent_data.ReplaySummary, string)
FileProcessingPipeline is performing the whole data processing pipeline for a replay file. Reads the replay, cleans the replay structure, creates replay summary, anonymizes, and creates a JSON replay output.
func MultiprocessingChunkPipeline ¶
func MultiprocessingChunkPipeline( listOfFiles []string, packageToZipBool bool, compressionMethod uint16, chunkIndex int, englishToForeignMapping map[string]string, progressBar *progressbar.ProgressBar, cliFlags utils.CLIFlags, singleJsonResultChan chan<- string, )
MultiprocessingChunkPipeline is a single instance of processing that is meant to be spawned by the orchestrator in order to speed up the process of data extraction.
Types ¶
type GRPCAnonymizer ¶
type GRPCAnonymizer struct {
Connection *grpc.ClientConn
Client pb.AnonymizeServiceClient
Cache map[string]string
}
Create new class, AnonymizerClient, that wraps the gRPC client (pb.NewAnonymizeServiceClient(conn) should happen once). The class will store the gRPC connection and can store a local cache of responses.
type ReplayProcessingChannelContents ¶
ReplayProcessingChannelContents is a struct that is used to pass data between the orchestrator and the workers in the pipeline.
type VerifyGameInfo ¶
type VerifyGameInfo struct {
// contains filtered or unexported fields
}
Source Files
¶
Directories
Click to show internal directories.
Click to hide internal directories.