יום שבת, 5 בדצמבר 2015

Disruptor Simple sample

The following is a simple sample of using Disruptor in order to deliver work to other threads

The Pom file:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>myLearn</groupId>
  <artifactId>Learn-disruptor</artifactId>
  <packaging>jar</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>disruptor-example</name>
  <url>http://maven.apache.org</url>
  <dependencies>
    <dependency>
      <groupId>com.googlecode.disruptor</groupId>
      <artifactId>disruptor</artifactId>
      <version>2.10.4</version>
    </dependency>
    <dependency>
      <groupId>junit</groupId>
      <artifactId>junit</artifactId>
      <version>3.8.1</version>
      <scope>test</scope>
    </dependency>
  </dependencies>
</project>

Declare a command that we wants to be executed using the Disruptor:
The ICommand


package myLearn;
public interface ICommand {
	public void DoWork ();
	public String getValue();
	public void setValue(String pValue);	
}

The concrete command


package myLearn;
public class DoThisAnThat implements ICommand {
	public String getValue() {
		return mValue;
	}
	public void setValue(String pValue) {
		this.mValue = pValue;
	}
	private String mValue ;
	
	public void DoWork() {
		 System.out.println(String.format("Call command for %s" ,mValue));	
	}
}

The command factory


package myLearn;
import com.lmax.disruptor.EventFactory;
public class DoThisAnThatFactory implements EventFactory<ICommand>
{	
	public ICommand newInstance() {
		
		return new DoThisAnThat();
	}
}

The program itself


package myLearn;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import com.lmax.disruptor.EventHandler;
import com.lmax.disruptor.RingBuffer;
import com.lmax.disruptor.dsl.Disruptor;
public class PlayWithdisruptor {
	public static void main(String[] args) {
		  ExecutorService exec = Executors.newCachedThreadPool();
	        // Preallocate RingBuffer with 4 ValueEvents
	        Disruptor<ICommand> disruptor = new Disruptor<ICommand>(new DoThisAnThatFactory(), 4, exec);
	    
	        final EventHandler<ICommand> handler = new EventHandler<ICommand>() {
	            // event will eventually be recycled by the Disruptor after it wraps
	            public void onEvent(final ICommand pCommand, final long sequence, final boolean endOfBatch) throws Exception {
	            	 System.out.println(String.format("the sequence %s theendOfBatch %s " , sequence,  Boolean.toString(endOfBatch)));
	            	pCommand.DoWork();
	            }
	        };
	        
	        disruptor.handleEventsWith(handler);
	        
	        RingBuffer<ICommand> ringBuffer = disruptor.start();
	        for (long i = 10; i < 20; i++) {
	            // Two phase commit. Grab one of the 4 slots
	            long seq = ringBuffer.next();
	            
	            ICommand theCommand = ringBuffer.get(seq);
	            
	            String theName = String.format("Hello Zvika (%d)", i);
	            
	            theCommand.setValue(theName);
	            
	            ringBuffer.publish(seq);
	            	         
	        }
	        System.out.println("After submiting the work");
	      
	        disruptor.shutdown();
	      
	        exec.shutdown();
	}
}

The Result
the sequence 0 theendOfBatch true
Call command for Hello Zvika (10)
the sequence 1 theendOfBatch false
Call command for Hello Zvika (11)
the sequence 2 theendOfBatch false
Call command for Hello Zvika (12)
the sequence 3 theendOfBatch true
Call command for Hello Zvika (13)
the sequence 4 theendOfBatch true
Call command for Hello Zvika (14)
the sequence 5 theendOfBatch true
Call command for Hello Zvika (15)
the sequence 6 theendOfBatch false
Call command for Hello Zvika (16)
the sequence 7 theendOfBatch false
Call command for Hello Zvika (17)
the sequence 8 theendOfBatch true
Call command for Hello Zvika (18)
After submiting the work
the sequence 9 theendOfBatch true
Call command for Hello Zvika (19)

יום שישי, 4 בדצמבר 2015

Tasks and coroutines in python 3.5

The following code demonstrates the using of python 3.5  coroutines in order to implement async IO

import asyncio
from threading import Thread, current_thread
import datetime
print (current_thread())
async def compute(x, y):
    print("Compute %s + %s ... Current thread:%s" % (x, y , current_thread() ))
  
    await asyncio.sleep(1.0)
    return x + y
async def print_sum(x, y):
    print (current_thread())
    result = await compute(x, y)
    print("%s + %s = %s" % (x, y, result))
loop = asyncio.get_event_loop()
theTask1 = loop.create_task(print_sum(1, 2))
theTask2 = loop.create_task(print_sum(4, 2))
theTask3 = loop.create_task(print_sum(14, 2))
tasks = [theTask1,theTask2,theTask3]
loop.run_until_complete(asyncio.wait(tasks))
loop.close()

And the result:
<_MainThread(MainThread, started 1696)>
<_MainThread(MainThread, started 1696)>
Compute 1 + 2 ... Current thread:<_MainThread(MainThread, started 1696)>
<_MainThread(MainThread, started 1696)>
Compute 4 + 2 ... Current thread:<_MainThread(MainThread, started 1696)>
<_MainThread(MainThread, started 1696)>
Compute 14 + 2 ... Current thread:<_MainThread(MainThread, started 1696)>
1 + 2 = 3
4 + 2 = 6
14 + 2 = 16


References
https://docs.python.org/3/library/asyncio-task.html
http://wla.berkeley.edu/~cs61a/fa11/lectures/streams.html#coroutines


 


using @ in Scala match case

There is an option In Scala match case to use Boolean extractors in match case .
The  unapply method  instead of returning an object returns Boolean expression.
The following example demonstrates it :

class  DataEntity (val mValue:Int) extends IDataEntity
object DataEntityValueChecker {
  def unapply(pIDataEntity: IDataEntity): Boolean = 
    {
      pIDataEntity.mValue > 164
    }
}
object MeshamemLiMavet10 extends App {
   println ("Starting MeshamemLiMavet 10 ")
    val theIDataEntity: IDataEntity = new DataEntity(165)
    theIDataEntity match {
        case pIDataEntity @ DataEntityValueChecker() => {println (pIDataEntity.mValue.toString() +  " > 164")}
        case _ => {println ( " > 164")}
   }

And the result:
Starting MeshamemLiMavet 10
165 > 164

Apache Ignite Simple Sample

Apache Ignite is a very powerful Grid memory DB .
The following simple sample do asynchronies  write and read from an Apache Ignite cache
Our test Project:
Capture1
The Pom file :

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>PlaywithIgnite</groupId>
  <artifactId>PlaywithIgnite</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <dependencies>
	<dependency>
		<groupId>org.apache.ignite</groupId>
		<artifactId>ignite-core</artifactId>
		<version>1.3.0-incubating</version>
	</dependency>
	<dependency>
		<groupId>org.apache.ignite</groupId>
		<artifactId>ignite-spring</artifactId>
		<version>1.3.0-incubating</version>
	</dependency>
	<dependency>
		<groupId>org.apache.ignite</groupId>
		<artifactId>ignite-indexing</artifactId>
		<version>1.3.0-incubating</version>
	</dependency>
</dependencies>
</project>

The entity that we are going to store and retrieve :


package PlayingArena;
import java.io.Serializable;
import org.apache.ignite.cache.query.annotations.QuerySqlField;
public class PersonData implements Serializable {
	  /** Will be visible in SQL. */
	  @QuerySqlField
	  private long id;
	  
	  /** Will be visible in SQL. */
	  @QuerySqlField
	  private String Name;
	  
	  @QuerySqlField
	  private int age;
	
	public PersonData(long id, String name, int age) {
		super();
		this.id = id;
		Name = name;
		this.age = age;
	}
	public PersonData() {
	
	}
		
	public String getName() {
		return Name;
	}
	public void setName(String mName) {
		this.Name = mName;
	}
	/* (non-Javadoc)
	 * @see java.lang.Object#toString()
	 */
	@Override
	public String toString() {
		return "PersonData [id=" + id + ", Name=" + Name + ", age=" + age + "]";
	}
	/**
	 * @return the id
	 */
	public long getId() {
		return id;
	}
	/**
	 * @param id the id to set
	 */
	public void setId(long id) {
		this.id = id;
	}
	/**
	 * @return the age
	 */
	public int getAge() {
		return age;
	}
	/**
	 * @param age the age to set
	 */
	public void setAge(int age) {
		this.age = age;
	}
	
}

The Ignite configuration File :


<?xml version="1.0" encoding="UTF-8"?>
<bean class="org.apache.ignite.configuration.IgniteConfiguration">
</bean>

A very  simple sample :


package PlayingArena;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
import java.util.concurrent.TimeUnit;
import javax.cache.Cache.Entry;
import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.IgniteException;
import org.apache.ignite.IgniteFileSystem;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.query.QueryCursor;
import org.apache.ignite.cache.query.ScanQuery;
import org.apache.ignite.cache.query.SqlQuery;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.FileSystemConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.igfs.IgfsPath;
import org.apache.ignite.lang.IgniteBiPredicate;
import org.apache.ignite.lang.IgniteClosure;
import org.apache.ignite.lang.IgniteFuture;
import org.apache.ignite.lang.IgniteInClosure;
public class StartPlaying {
		
	public static void main(String[] args) throws Exception {
			
		Ignite ignite = Ignition.start("example-ignite.xml");
		
		CacheConfiguration<?, ?> cfg = new CacheConfiguration<Integer, PersonData>();
		cfg.setName("myCache");
		
		// Create cache with given name, if it does not exist.
		IgniteCache<Integer, PersonData> cache = (IgniteCache<Integer, PersonData>) ignite.getOrCreateCache(cfg);
	
		  // Store keys in cache (values will end up on different cache nodes).
	    for (int i = 0; i < 10000 ; i++)
	        cache.put(i, new PersonData (i , Integer.toString(i) ,i * 10));
	 
	    for (int i = 0; i < 10; i++)
	        System.out.println("Got [key=" + i* 164 + ", val=" + cache.get(i) + ']');

The Result


 

Dec 04, 2015 12:23:54 PM java.util.logging.LogManager$RootLogger log
SEVERE: Failed to resolve default logging config file: config/java.util.logging.properties
[12:23:54]    __________  ________________ 
[12:23:54]   /  _/ ___/ |/ /  _/_  __/ __/ 
[12:23:54]  _/ // (7 7    // /  / / / _/   
[12:23:54] /___/\___/_/|_/___/ /_/ /___/  
[12:23:54] 
[12:23:54] ver. 1.3.0-incubating#20150710-sha1:2ade6d00
[12:23:54] 2015 Copyright(C) Apache Software Foundation
[12:23:54] 
[12:23:54] Ignite documentation: http://ignite.incubator.apache.org
[12:23:54] 
[12:23:54] Quiet mode.
[12:23:54]   ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-v" to ignite.{sh|bat}
[12:23:54] 
[12:23:56] Configured plugins:
[12:23:56]   ^-- None
[12:23:56] 
[12:24:06] New version is available at ignite.incubator.apache.org: 1.4.0
[12:24:16] Performance suggestions for grid  (fix if possible)
[12:24:16] To disable, set -DIGNITE_PERFORMANCE_SUGGESTIONS_DISABLED=true
[12:24:16]   ^-- Disable peer class loading (set 'peerClassLoadingEnabled' to false)
[12:24:16]   ^-- Disable grid events (remove 'includeEventTypes' from configuration)
[12:24:16] 
[12:24:16] To start Console Management & Monitoring run ignitevisorcmd.{sh|bat}
[12:24:16] 
[12:24:16] Ignite node started OK (id=5363064c)
[12:24:16] Topology snapshot [ver=1, server nodes=1, client nodes=0, CPUs=4, heap=0.5GB]
Got [key=0, val=PersonData [id=0, Name=0, age=0]]
Got [key=164, val=PersonData [id=1, Name=1, age=10]]
Got [key=328, val=PersonData [id=2, Name=2, age=20]]
Got [key=492, val=PersonData [id=3, Name=3, age=30]]
Got [key=656, val=PersonData [id=4, Name=4, age=40]]
Got [key=820, val=PersonData [id=5, Name=5, age=50]]
Got [key=984, val=PersonData [id=6, Name=6, age=60]]
Got [key=1148, val=PersonData [id=7, Name=7, age=70]]
Got [key=1312, val=PersonData [id=8, Name=8, age=80]]
Got [key=1476, val=PersonData [id=9, Name=9, age=90]]
Previous cache value: PersonData [id=4, Name=4, age=40]

יום רביעי, 21 במאי 2014

Ruby define_method simple sample‏

The following Ruby code demonstrates the use of rubies define_method in order to define methods dynamically.

class MyCls
      %w(getData1 getData2).each do |meth|
      define_method(meth) {
        @data[meth.to_sym] = @data[:User] * 2
        @data[meth.to_sym]
        }
  def initialize()
    @data = {}
    @data[:User]= 15 
    end
  end
end
theMyCls = MyCls.new()
puts  theMyCls.getData2
 

Resources
#http://ruby-doc.org/core-2.0/Module.html#method-i-define_method
#http://www.trottercashion.com/2011/02/08/rubys-define_method-method_missing-and-instance_eval.html

יום שלישי, 20 במאי 2014

Zero MQ Pub Sub Simple Sample

The following is a simple 0MQ tcp pub sub simple sample:
The main:

        ZMQ.Context context = ZMQ.context(1);
   
        ZeroMqPubServer theZeroMqPubServer = new ZeroMqPubServer(context);
        theZeroMqPubServer.start();
      
        ZeroMqSubClient theZeroMqSubClient = new ZeroMqSubClient(context);
        
        theZeroMqSubClient.start();
The server

import java.util.logging.Level;
import java.util.logging.Logger;
import org.zeromq.ZMQ;
public class ZeroMqPubServer  extends Thread{
    
    ZMQ.Context  mContext;
    public ZeroMqPubServer(ZMQ.Context pContext) {
        this.mContext = pContext;
    }
    
     @Override
     public void run()
     {
         ZMQ.Socket publisher = mContext.socket(ZMQ.PUB);
        publisher.bind("tcp://*:5556");
        int theCounter = 0 ; 
         while (!Thread.currentThread ().isInterrupted ()) {
            try {
                theCounter++;
                //  Send message to all subscribers
                String update = String.format("MyTopic %d ", theCounter);
                
                publisher.send(update, 0);
                
                Thread.sleep(100);
            }
            // context.term ();
            catch (InterruptedException ex) {
                Logger.getLogger(ZeroMqPubServer.class.getName()).log(Level.SEVERE, null, ex);
            }          
        }
     }
}

The client


import java.nio.charset.Charset;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.zeromq.ZMQ;
public class ZeroMqSubClient  extends Thread{
    
      ZMQ.Context  mContext;
    public ZeroMqSubClient(ZMQ.Context pContext) {
        this.mContext = pContext;
    }
    
    
     @Override
     public void run()
     {
          try {
              Thread.sleep(3000 );
              
              ZMQ.Socket subscriber = mContext.socket(ZMQ.SUB);
              subscriber.connect("tcp://localhost:5556");
              
              //  Subscribe to zipcode, default is NYC, 10001
              String filter =  "MyTopic";
              subscriber.subscribe(filter.getBytes());
              
              while (!Thread.currentThread ().isInterrupted ()) {
                  
                  //  Use trim to remove the tailing '0' character
                  String theStr = subscriber.recvStr(Charset.defaultCharset());
                  
                  System.out.println("Recieved:" + theStr);
              }
              subscriber.close();
              //  mContext.term();
          } catch (InterruptedException ex) {
              Logger.getLogger(ZeroMqSubClient.class.getName()).log(Level.SEVERE, null, ex);
          }
    }
}

The result
debug:
Start
Recieved:MyTopic 31
Recieved:MyTopic 32
Recieved:MyTopic 33
Recieved:MyTopic 34
Recieved:MyTopic 35
Recieved:MyTopic 36
Recieved:MyTopic 37
Recieved:MyTopic 38
Recieved:MyTopic 39
Recieved:MyTopic 40
Recieved:MyTopic 41
Recieved:MyTopic 42
Recieved:MyTopic 43
Recieved:MyTopic 44
Recieved:MyTopic 45
Recieved:MyTopic 46
Recieved:MyTopic 47
Recieved:MyTopic 48
Recieved:MyTopic 49
Recieved:MyTopic 50
Recieved:MyTopic 51
Recieved:MyTopic 52
Recieved:MyTopic 53
Recieved:MyTopic 54
Recieved:MyTopic 55