View on GitHub

Spark-flint

Small utility library for apache spark

data distribution
alias example
query example
describe table
repl show source
Download this project as a .zip file Download this project as a tar.gz file

Quickly plot some data

You can map RDDs as table and run sql queries from the WebUI at http://localhost:4040/plugins. Queries can be aliased, data distribution can be visualized with simple commands like

Check aliases command for a list of aliases and help for general help.

Refer to README.md  for sample and how to start

Built-In components

Add a new component

Useful classes and methods are grouped together as components like the code-analysis component. Components are loaded using :load <comp-location> , the <comp-name>.init file in turn loads the needed classes and invokes handy methods.