יום שבת, 5 בדצמבר 2015

Disruptor Simple sample

The following is a simple sample of using Disruptor in order to deliver work to other threads

The Pom file:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>myLearn</groupId>
  <artifactId>Learn-disruptor</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>disruptor-example</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>com.googlecode.disruptor</groupId>
      <artifactId>disruptor</artifactId>
      <version>2.10.4</version>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
</project>

Declare a command that we wants to be executed using the Disruptor:
The ICommand


package myLearn;
public interface ICommand {
	public void DoWork ();
	public String getValue();
	public void setValue(String pValue);	
}

The concrete command


package myLearn;
public class DoThisAnThat implements ICommand {
	public String getValue() {
		return mValue;
	}
	public void setValue(String pValue) {
		this.mValue = pValue;
	}
	private String mValue ;
	
	public void DoWork() {
		 System.out.println(String.format("Call command for %s" ,mValue));	
	}
}

The command factory


package myLearn;
import com.lmax.disruptor.EventFactory;
public class DoThisAnThatFactory implements EventFactory<ICommand>
{	
	public ICommand newInstance() {
		
		return new DoThisAnThat();
	}
}

The program itself


package myLearn;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import com.lmax.disruptor.EventHandler;
import com.lmax.disruptor.RingBuffer;
import com.lmax.disruptor.dsl.Disruptor;
public class PlayWithdisruptor {
	public static void main(String[] args) {
		  ExecutorService exec = Executors.newCachedThreadPool();
	        // Preallocate RingBuffer with 4 ValueEvents
	        Disruptor<ICommand> disruptor = new Disruptor<ICommand>(new DoThisAnThatFactory(), 4, exec);
	    
	        final EventHandler<ICommand> handler = new EventHandler<ICommand>() {
	            // event will eventually be recycled by the Disruptor after it wraps
	            public void onEvent(final ICommand pCommand, final long sequence, final boolean endOfBatch) throws Exception {
	            	 System.out.println(String.format("the sequence %s theendOfBatch %s " , sequence,  Boolean.toString(endOfBatch)));
	            	pCommand.DoWork();
	            }
	        };
	        
	        disruptor.handleEventsWith(handler);
	        
	        RingBuffer<ICommand> ringBuffer = disruptor.start();
	        for (long i = 10; i < 20; i++) {
	            // Two phase commit. Grab one of the 4 slots
	            long seq = ringBuffer.next();
	            
	            ICommand theCommand = ringBuffer.get(seq);
	            
	            String theName = String.format("Hello Zvika (%d)", i);
	            
	            theCommand.setValue(theName);
	            
	            ringBuffer.publish(seq);
	            	         
	        }
	        System.out.println("After submiting the work");
	      
	        disruptor.shutdown();
	      
	        exec.shutdown();
	}
}

The Result
the sequence 0 theendOfBatch true
Call command for Hello Zvika (10)
the sequence 1 theendOfBatch false
Call command for Hello Zvika (11)
the sequence 2 theendOfBatch false
Call command for Hello Zvika (12)
the sequence 3 theendOfBatch true
Call command for Hello Zvika (13)
the sequence 4 theendOfBatch true
Call command for Hello Zvika (14)
the sequence 5 theendOfBatch true
Call command for Hello Zvika (15)
the sequence 6 theendOfBatch false
Call command for Hello Zvika (16)
the sequence 7 theendOfBatch false
Call command for Hello Zvika (17)
the sequence 8 theendOfBatch true
Call command for Hello Zvika (18)
After submiting the work
the sequence 9 theendOfBatch true
Call command for Hello Zvika (19)

יום שישי, 4 בדצמבר 2015

Tasks and coroutines in python 3.5

The following code demonstrates the using of python 3.5  coroutines in order to implement async IO

import asyncio
from threading import Thread, current_thread
import datetime
print (current_thread())
async def compute(x, y):
    print("Compute %s + %s ... Current thread:%s" % (x, y , current_thread() ))
  
    await asyncio.sleep(1.0)
    return x + y
async def print_sum(x, y):
    print (current_thread())
    result = await compute(x, y)
    print("%s + %s = %s" % (x, y, result))
loop = asyncio.get_event_loop()
theTask1 = loop.create_task(print_sum(1, 2))
theTask2 = loop.create_task(print_sum(4, 2))
theTask3 = loop.create_task(print_sum(14, 2))
tasks = [theTask1,theTask2,theTask3]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

And the result:
<_MainThread(MainThread, started 1696)>
<_MainThread(MainThread, started 1696)>
Compute 1 + 2 ... Current thread:<_MainThread(MainThread, started 1696)>
<_MainThread(MainThread, started 1696)>
Compute 4 + 2 ... Current thread:<_MainThread(MainThread, started 1696)>
<_MainThread(MainThread, started 1696)>
Compute 14 + 2 ... Current thread:<_MainThread(MainThread, started 1696)>
1 + 2 = 3
4 + 2 = 6
14 + 2 = 16


References
https://docs.python.org/3/library/asyncio-task.html
http://wla.berkeley.edu/~cs61a/fa11/lectures/streams.html#coroutines


 


using @ in Scala match case

There is an option In Scala match case to use Boolean extractors in match case .
The  unapply method  instead of returning an object returns Boolean expression.
The following example demonstrates it :

class  DataEntity (val mValue:Int) extends IDataEntity
object DataEntityValueChecker {
  def unapply(pIDataEntity: IDataEntity): Boolean = 
    {
      pIDataEntity.mValue > 164
    }
}
object MeshamemLiMavet10 extends App {
   println ("Starting MeshamemLiMavet 10 ")
    val theIDataEntity: IDataEntity = new DataEntity(165)
    theIDataEntity match {
        case pIDataEntity @ DataEntityValueChecker() => {println (pIDataEntity.mValue.toString() +  " > 164")}
        case _ => {println ( " > 164")}
   }

And the result:
Starting MeshamemLiMavet 10
165 > 164

Apache Ignite Simple Sample

Apache Ignite is a very powerful Grid memory DB .
The following simple sample do asynchronies  write and read from an Apache Ignite cache
Our test Project:
Capture1
The Pom file :

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>PlaywithIgnite</groupId>
  <artifactId>PlaywithIgnite</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <dependencies>
	<dependency>
		<groupId>org.apache.ignite</groupId>
		<artifactId>ignite-core</artifactId>
		<version>1.3.0-incubating</version>
	</dependency>
	<dependency>
		<groupId>org.apache.ignite</groupId>
		<artifactId>ignite-spring</artifactId>
		<version>1.3.0-incubating</version>
	</dependency>
	<dependency>
		<groupId>org.apache.ignite</groupId>
		<artifactId>ignite-indexing</artifactId>
		<version>1.3.0-incubating</version>
	</dependency>
</dependencies>
</project>

The entity that we are going to store and retrieve :


package PlayingArena;
import java.io.Serializable;
import org.apache.ignite.cache.query.annotations.QuerySqlField;
public class PersonData implements Serializable {
	  /** Will be visible in SQL. */
	  @QuerySqlField
	  private long id;
	  
	  /** Will be visible in SQL. */
	  @QuerySqlField
	  private String Name;
	  
	  @QuerySqlField
	  private int age;
	
	public PersonData(long id, String name, int age) {
		super();
		this.id = id;
		Name = name;
		this.age = age;
	}
	public PersonData() {
	
	}
		
	public String getName() {
		return Name;
	}
	public void setName(String mName) {
		this.Name = mName;
	}
	/* (non-Javadoc)
	 * @see java.lang.Object#toString()
	 */
	@Override
	public String toString() {
		return "PersonData [id=" + id + ", Name=" + Name + ", age=" + age + "]";
	}
	/**
	 * @return the id
	 */
	public long getId() {
		return id;
	}
	/**
	 * @param id the id to set
	 */
	public void setId(long id) {
		this.id = id;
	}
	/**
	 * @return the age
	 */
	public int getAge() {
		return age;
	}
	/**
	 * @param age the age to set
	 */
	public void setAge(int age) {
		this.age = age;
	}
	
}

The Ignite configuration File :


<?xml version="1.0" encoding="UTF-8"?>
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
</bean>

A very  simple sample :


package PlayingArena;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
import java.util.concurrent.TimeUnit;
import javax.cache.Cache.Entry;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.IgniteException;
import org.apache.ignite.IgniteFileSystem;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.query.QueryCursor;
import org.apache.ignite.cache.query.ScanQuery;
import org.apache.ignite.cache.query.SqlQuery;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.FileSystemConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.igfs.IgfsPath;
import org.apache.ignite.lang.IgniteBiPredicate;
import org.apache.ignite.lang.IgniteClosure;
import org.apache.ignite.lang.IgniteFuture;
import org.apache.ignite.lang.IgniteInClosure;
public class StartPlaying {
		
	public static void main(String[] args) throws Exception {
			
		Ignite ignite = Ignition.start("example-ignite.xml");
		
		CacheConfiguration<?, ?> cfg = new CacheConfiguration<Integer, PersonData>();
		cfg.setName("myCache");
		
		// Create cache with given name, if it does not exist.
		IgniteCache<Integer, PersonData> cache = (IgniteCache<Integer, PersonData>) ignite.getOrCreateCache(cfg);
	
		  // Store keys in cache (values will end up on different cache nodes).
	    for (int i = 0; i < 10000 ; i++)
	        cache.put(i, new PersonData (i , Integer.toString(i) ,i * 10));
	 
	    for (int i = 0; i < 10; i++)
	        System.out.println("Got [key=" + i* 164 + ", val=" + cache.get(i) + ']');

The Result


 

Dec 04, 2015 12:23:54 PM java.util.logging.LogManager$RootLogger log
SEVERE: Failed to resolve default logging config file: config/java.util.logging.properties
[12:23:54]    __________  ________________ 
[12:23:54]   /  _/ ___/ |/ /  _/_  __/ __/ 
[12:23:54]  _/ // (7 7    // /  / / / _/   
[12:23:54] /___/\___/_/|_/___/ /_/ /___/  
[12:23:54] 
[12:23:54] ver. 1.3.0-incubating#20150710-sha1:2ade6d00
[12:23:54] 2015 Copyright(C) Apache Software Foundation
[12:23:54] 
[12:23:54] Ignite documentation: http://ignite.incubator.apache.org
[12:23:54] 
[12:23:54] Quiet mode.
[12:23:54]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
[12:23:54] 
[12:23:56] Configured plugins:
[12:23:56]   ^-- None
[12:23:56] 
[12:24:06] New version is available at ignite.incubator.apache.org: 1.4.0
[12:24:16] Performance suggestions for grid  (fix if possible)
[12:24:16] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
[12:24:16]   ^-- Disable peer class loading (set 'peerClassLoadingEnabled' to false)
[12:24:16]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
[12:24:16] 
[12:24:16] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
[12:24:16] 
[12:24:16] Ignite node started OK (id=5363064c)
[12:24:16] Topology snapshot [ver=1, server nodes=1, client nodes=0, CPUs=4, heap=0.5GB]
Got [key=0, val=PersonData [id=0, Name=0, age=0]]
Got [key=164, val=PersonData [id=1, Name=1, age=10]]
Got [key=328, val=PersonData [id=2, Name=2, age=20]]
Got [key=492, val=PersonData [id=3, Name=3, age=30]]
Got [key=656, val=PersonData [id=4, Name=4, age=40]]
Got [key=820, val=PersonData [id=5, Name=5, age=50]]
Got [key=984, val=PersonData [id=6, Name=6, age=60]]
Got [key=1148, val=PersonData [id=7, Name=7, age=70]]
Got [key=1312, val=PersonData [id=8, Name=8, age=80]]
Got [key=1476, val=PersonData [id=9, Name=9, age=90]]
Previous cache value: PersonData [id=4, Name=4, age=40]