on 17-Feb-2020 (Mon)

The branch you added your changes into is called source branch while the branch you request to merge your changes into is called target branch.

Abstract: Graph classification is a difficult problem that has drawn a lot of attention from the machine learning community over the past few years. This is mainly due to the fact that, contrarily to Euclidean vectors, the inherent complexity of graph structures can be quite hard to encode and handle for traditional classifiers. Even though kernels have been proposed in the literature, the increase in the dataset sizes has greatly limited the use of kernel methods since computation and storage of kernel matrices has become impracticable. In this article, we propose to use extended persistence diagrams to efficiently encode graph structure. More precisely, we show that using the so-called heat kernel signatures for the computation of these extended persistence diagrams allows one to quickly and efficiently summarize the graph structure. Then, we build on the recent development of neural networks for point clouds to define an architecture for (extended) persistence diagrams which is modular and easy-to-use. Finally, we demonstrate the usefulness of our approach by validating our architecture on several graph datasets, on which the obtained results are comparable to the state-of-the-art for graph classification.

Abstract: Graph Neural Nets (GNNs) have received increasing attentions, partially due to their superior performance in many node and graph classification tasks. However, there is a lack of understanding on what they are learning and how sophisticated the learned graph functions are. In this work, we first propose Graph Feature Network (GFN), a simple lightweight neural net defined on a set of graph augmented features. We then propose a dissection of GNNs on graph classification into two parts: 1) the graph filtering, where graph-based neighbor aggregations are performed, and 2) the set function, where a set of hidden node features are composed for prediction. We prove that GFN can be derived by linearizing graph filtering part of GNNs, and leverage it to test the importance of the two parts separately. Empirically we perform evaluations on common graph classification benchmarks. To our surprise, we find that, despite the simplification, GFN could match or exceed the best accuracies produced by recently proposed GNNs, with a fraction of computation cost. Our results suggest that linear graph filtering with non-linear set function is powerful enough, and common graph classification benchmarks seem inadequate for testing advanced GNN variants.

Abstract: The ability of a graph neural network (GNN) to leverage both the graph topology and graph labels is fundamental to building discriminative node and graph embeddings. Building on previous work, we theoretically show that edGNN, our model for directed labeled graphs, is as powerful as the Weisfeiler-Lehman algorithm for graph isomorphism. Our experiments support our theoretical findings, confirming that graph neural networks can be used effectively for inference problems on directed graphs with both node and edge labels. Code available at this https URL.

Preprints

Ordering-based causal structure learning in the presence of latent variables
D. I. Bernstein, B. Saeed, C. Squires and C. Uhler

Under review

[ arXiv ]

Keywords: causal inference, algebraic statistics

​

Permutation-based causal structure learning with unknown intervention targets
C. Squires, Y. Wang and C. Uhler

Under review

[ arXiv ]

Keywords: causal inference, applications to biology

​

Overparameterized neural networks can implement associative memory
A. Radhakrishnan, M. Belkin and C. Uhler

Under review

[ arXiv ]

Keywords: autoencoders, machine learning

​

Covariance matrix estimation under total positivity for portfolio selection
R. Agrawal, U. Roy and C. Uhler

Under review

[ arXiv ]

Keywords: total positivity, applications to finance

​

Learning high-dimensional Gaussian graphical models under total positivity without tuning parameters
Y. Wang, U. Roy and C. Uhler

Under review

[ arXiv ]

Keywords: total positivity, applications to finance

​

Anchored causal inference in the presence of measurement error
B. Saeed, A. Belyaeva, Y. Wang and C. Uhler

Under review

[ arXiv ]

Keywords: causal inference, applications to biology

​

Total positivity in structured binary distributions
S. Lauritzen, C. Uhler and P. Z

2019

Algebraic statistics in practice: Applications to networks
M. Casanellas, S. Petrovic and C. Uhler

To appear in Annual Review of Statistics and its Applications (invited review)

[ arXiv ]

Keywords: causal inference, algebraic statistics

​

Maximum likelihood estimation in Gaussian models under total positivity
S. Lauritzen, C. Uhler and P. Zwiernik

Annals of Statistics 47 (2019), pp. 1835-1863.

[ arXiv ]

Keywords: total positivity, Gaussian graphical models

​

Multi-domain translation by learning uncoupled autoencoders
K.D. Yang and C. Uhler

Computational Biology Workshop, International Conference on Machine Learning (ICML 2019)

[ arXiv ]

Keywords: autoencoders, machine learning

​

Memorization in overparameterized autoencoders
A. Radhakrishnan, K.D. Yang, M. Belkin and C. Uhler

Deep Phenomena Workshop, International Conference on Machine Learning (ICML 2019)

[ arXiv ]

Keywords: autoencoders, machine learning

​

ABCD-Strategy: Budgeted experimental design for targeted causal structure discovery
R. Agrawal, C. Squires, K.D. Yang, K. Shanmugam and C. Uhler

Proceedings of Machine Learning Research 89 (AISTATS 2019), pp. 3400-3409

Keywords: causal inference, experimental design

​

Size of interventional Markov equivalence classes in random DAG models

D. Katz-Rogozhnikov,K. Shanmugam, C. Squires and C. Uhler

Proceedings of Machine Learning Research 89 (AISTATS 2019), pp. 3234-3243

Keywords: causal inference

2017

Machine learning for nuclear mechano-morphometric biomarkers in cancer diagnosis
A. Radhakrishnan, D. Damodaran, A. Soylemezoglu, C. Uhler, and G.V. Shivashankar

Scientific Reports 7 (2017), article nr. 17946.

[ journal ]

Keywords: neural nets, applications to biology

​

Permutation-based causal inference algorithms with interventions

Y. Wang, L. Solus, K.D. Yang and C. Uhler

Advances in Neural Information Processing (NIPS 2017)

[ arXiv ]

Keywords: causal inference, algebraic statistics

​

Network analysis identifies chromosome intermingling regions as regulatory hotspots for transcription
A. Belyaeva, S. Venkatachalapathy, M. Nagarajan, G.V. Shivashankar and C. Uhler

Proceedings of the National Academy of Sciences, U.S.A. 114 (2017), pp. 13714-13719.

[ journal ]

Keywords: chromosome packing, applications to biology

​

Regulation of genome organization and gene expression by nuclear mechanotransduction
C. Uhler and G.V. Shivashankar

Nature Reviews Molecular Cell Biology 18 (2017), pp. 717-727 (invited review).

[ journal ]

Keywords: applications to biology, chromosome packing

​

Chromoso

2013

Packing ellipsoids with overlap

C. Uhler and S.J. Wright

SIAM Review 55 (2013), pp. 671-706 (selected as Research Spotlight)

[ arXiv ]

Keywords: optimization, sphere packing, chromosome packing

​

Geometry of faithfulness assumption in causal inference

C. Uhler, G. Raskutti, P. Bühlmann and B. Yu

Annals of Statistics 41 (2013), pp. 436-463

[ arXiv ]

Keywords: causal inference, algebraic statistics

Privacy-preserving data sharing for genome-wide association studies

C. Uhler, S.E. Fienberg and A. Slavkovic

Journal of Privacy and Confidentiality 5 (2013), pp. 137-166

[ arXiv ]

Keywords: differential privacy, applications to biology

​

​

2012

Geometry of maximum likelihood estimation in Gaussian graphical models

C. Uhler

Annals of Statistics 40 (2012), pp. 238-261

[ arXiv ]

Keywords: Gaussian graphical models, algebraic statistics

​

​

2011

Privacy preserving GWAS data sharing

It depends on what are you trying to accomplish.

DVC will help you with organizing the ML experimentation process.

Pachyderm is more focused on data engineering pipelines.

Here’s a summary of those efforts (both native Kubeflow projects and related K8s integration with Kubeflow):

• Argo for managing ML workflows
• Arrikto’s product integration with Kubeflow, for collaboration and workflows that span hybrid and multi-cloud environments
• Caffe2 Operator for running Caffe2 jobs
• Horovod & MPI Operator for improved distributed training performance of TensorFlow
• Istio Integration for TFServing
• Kubeflow Pipelines for tooling to compose, deploy and manage end-to-end machine learning workflows.
• Katib for hyperparameter tuning
• Kubebench for benchmarking of HW and ML stacks
• Kubeflow on MicroK8s enabling Mac users to easily install default Kubeflow components on a single-node Kubernetes cluster running in a native MacOS hypervisor
• MXNet Operator for running MXNet Jobs
• Nvidia Rapids to accelerate Kubeflow pipeline with GPUs on Kubernetes.
• PyTorch operator for running PyTorch jobs
• Pachyderm for managing complex data pipelines
• Seldon Core for running complex model deployments and non-TensorFlow serving

Containers are a good way to bundle and run your applications. In a production environment, you need to manage the containers that run the applications and ensure that there is no downtime. For example, if a container goes down, another container needs to start. Wouldn’t it be easier if this behavior was handled by a system?

That’s how Kubernetes comes to the rescue! Kubernetes provides you with a framework to run distributed systems resiliently. It takes care of scaling and failover for your application, provides deployment patterns, and more

As part of our on-going bash tutorial series, we discussed about bash positional parameters in our previous article. In this article let us discuss about the bash special parameters with few practical shell script examples.

Some of the bash special parameters that we will discuss in this article are: $*, $@, $#, $$, $!, $?, $-, $_

To access the whole list of positional parameters, the two special parameters $* and $@ are available. Outside of double quotes, these two are equivalent: Both expand to the list of positional parameters starting with $1 (separated by spaces).

Within double quotes, however, they differ: $* within a pair of double quotes is equivalent to the list of positional parameters, separated by the first character of IFS “$1c$2c$3…”.

$@ within a pair of double quotes is equivalent to the list of positional parameters, separated by unquoted spaces, i.e., “$1” “$2″..”$N”.

Example 1: Use Bash $* and $@ to Expand Positional Parameters

This example shows the value available in $* and $@.

First, create the expan.sh as shown below.

$ cat expan.sh
#!/bin/bash

export IFS='-'

cnt=1

# Printing the data available in $*
echo "Values of \"\$*\":"
for arg in "$*"
do
 echo "Arg #$cnt= $arg"
 let "cnt+=1"
done

cnt=1

# Printing the data available in $@
echo "Values of \"\$@\":"
for arg in "$@"
do
 echo "Arg #$cnt= $arg"
 let "cnt+=1"
done

Next, execute the expan.sh as shown below to see how $* and $@ works.

$ ./expan.sh "This is" 2 3
Values of "$*":
Arg #1= This is-2-3
Values of "$@":
Arg #1= This is
Arg #2= 2
Arg #3= 3

• The above script exported the value of IFS (Internal Field Separator) with the ‘-‘.
• There are three parameter passed to the script expan.sh $1=”This is”,$2=”2″ and $3=”3″.
• When printing the each value of special parameter “$*”, it gives only one value which is the whole positional parameter delimited by IFS.
• Whereas “$@” gives you each parameter as a separate word.

Example 2: Use $# to Count Positional Parameters

$# is the special parameter in bash which gives you the number of positional parameter in decimal.

First, create the arithmetic.sh as shown below.

$ cat arithmetic.sh
#!/bin/bash

if [ $# -lt 2 ]
then
 echo "Usage: $0 arg1 arg2"
 exit
fi

echo -e "\$1=$1"
echo -e "\$2=$2"

let add=$1+$2
let sub=$1-$2
let mul=$1*$2
let div=$1/$2

echo -e "Addition=$add\nSubtraction=$sub\nMultiplication=$mul\nDivision=$div\n"

If the number of positional parameters is less than 2, it will throw the usage information as shown below,

$ ./arithemetic.sh 10
Usage: ./arithemetic.sh arg1 arg2

Example 3: Process related Parameters – $$ and $!

The special parameter $$ will give the process ID of the shell. $! gives you the process id of the most recently executed background process.

The following script prints the process id of the shell and last execute background process ID.

$ cat proc.sh
#!/bin/bash

echo -e "Process ID=$$"

sleep 1000 &

echo -e "Background Process ID=$!"

Now, execute the above script, and check the process id which its printing.

$ ./proc.sh
Process ID=9502
Background Process ID=9503
$ ps
 PID TTY TIME CMD
 5970 pts/1 00:00:00 bash
 9503 pts/1 00:00:00 sleep
 9504 pts/1 00:00:00 ps
$

Example 4: Other Bash Special Parameters – $?, $-, $_

• $? Gives the exit status of the most recently executed command.
• $- Option

common functional silos in large organizations can create barriers, stifling the ability to automate the end-to-end process of deploying ML applications to production

This leads to delays and friction. A common symptom is having models that only work in a lab environment and never leave the proof-of-concept phase. Or if they make it to production, in a manual ad-hoc way, they become stale and hard to update.

In order to formalise the model training process in code, we used an open source tool called DVC (Data Science Version Control). It provides similar semantics to Git, but also solves a few ML-specific problems:

• it has multiple backend plugins to fetch and store large files on an external storage outside of the source control repository;
• it can keep track of those files' versions, allowing us to retrain our models when the data changes;
• it keeps track of the dependency graph and commands used to execute the ML pipeline, allowing the process to be reproduced in other environments;
• it can integrate with Git branches to allow multiple experiments to co-exist;

For example, we can configure our initial ML pipeline in Figure 5 with three dvc run commands (-d specify dependencies, -o specify outputs, -f is the filename to record that step, and -M is the resulting metrics):

dvc run -f input.dvc \ ➊
 -d src/download_data.py -o data/raw/store47-2016.csv python src/download_data.py
dvc run -f split.dvc \ ➋
 -d data/raw/store47-2016.csv -d src/splitter.py \
 -o data/splitter/train.csv -o data/splitter/validation.csv python src/splitter.py
dvc run ➌
 -d data/splitter/train.csv -d data/splitter/validation.csv -d src/decision_tree.py \
 -o data/decision_tree/model.pkl -M results/metrics.json python src/decision_tree.py

Each run will create a corresponding file, that can be committed to version control, and that allows other people to reproduce the entire ML pipeline, by executing the dvc repro command.

Once we find a suitable model, we will treat it as an artifact that needs to be versioned and deployed to production. With DVC, we can use the dvc push and dvc pull commands to publish and fetch it from external storage.