Play with Payton collections

Derived my class from dict and overriding the __missing__ method used to return value in case the key is not in the collection.

  1: class myTestCls (dict):
  2:     def __missing__ (self, key):
  3:         return "Not here sory"
  4: theCls = myTestCls();
  5: 
  6: print ( theCls["TestValue"])
  7: 
  8: print (" -- done --") 
  9: 
 10: 

Check if the new type collection is equal to base collection with the same values

  1: class myTestCls (dict):
  2:     def __missing__ (self, key):
  3:         return "Not here sory"
  4: 
  5: theCls1 = myTestCls(cat=1, dog=2, pig=3)
  6: 
  7: theCls2 = {'cat': 1, 'dog': 2, 'pig': 3} 
  8: 
  9: print ( theCls1 == theCls2)
 10: 
 11: print ( type ( theCls1))
 12: 
 13: print ( type ( theCls2))
 14: 
 15: print (" -- done --") 
 16: 

The results
True
<class '__main__.myTestCls'>
<class 'dict'>

Override the __eq__ method in order to check that the types of the objects are the same.

  1: class myTestCls (dict):
  2:     def __missing__ (self, key):
  3:         return "Not here sory"
  4:     def __eq__(self, other):
  5:         if type (other) == type( self):
  6:             return super(myTestCls, self).__eq__( other)
  7:         return False
  8:         
  9:     def __ne__(self, other):
 10:         return NotImplemented
 11: 
 12: theCls1 = myTestCls(cat=1, dog=2, pig=3)
 13: 
 14: theCls2 = {'cat': 1, 'dog': 2, 'pig': 3} 
 15: 
 16: theCls3 = myTestCls(zip (['cat', 'dog', 'pig'], [1, 2, 3]))
 17: 
 18: print ( theCls1 == theCls2)
 19: 
 20: print ( theCls1 == theCls3)
 21: 
 22: print ( type ( theCls1))
 23: 
 24: print ( type ( theCls2))
 25: 
 26: print ( type ( theCls3))
 27: 
 28: print (" -- done –")

returns
False
True
<class '__main__.myTestCls'>
<class 'dict'>
<class '__main__.myTestCls'>

http://docs.python.org/3.3/library/stdtypes.html#set

usbfx2 device probing in linux

I wanted to exercise my USB device driver writing in Linux.
I wrote the following Skelton of USB device for my FX2 USB card

  1: #include <linux/kernel.h>    /* We're doing kernel work */ 
  2: #include <linux/module.h>    /* Specifically, a module */ 
  3: #include <linux/fs.h> 
  4: #include <asm/uaccess.h>    /* for get_user and put_user */ 
  5: #include  <linux/slab.h> 
  6: #include <linux/timer.h> 
  7: #include <linux/mod_devicetable.h> 
  8: #include <linux/usb.h> 
  9:  
 10: #define SUCCESS 0 
 11: 
 12: #define DEBUG   
 13:   
 14: /*  
 15:  * Is the device open right now? Used to prevent 
 16:  * concurent access into the same device  
 17:  * http://www.osronline.com/hardware/osrfx2_32.pdf 
 18:  */ 
 19:  int Device_Open = 0; 
 20:  
 21: #define ML_VENDOR_ID 0x0547 
 22: #define ML_PRODUCT_ID 0x1002 
 23:   
 24: static struct usb_device_id ml_table [] = { 
 25:     { USB_DEVICE(ML_VENDOR_ID, ML_PRODUCT_ID) }, 
 26:     { } 
 27: }; 
 28: 
 29: MODULE_DEVICE_TABLE(usb, ml_table); 
 30: 
 31: static int ml_probe(struct usb_interface *interface, const struct usb_device_id  *id) 
 32: { 
 33:     /* called when a USB device is connected to the computer. */ 
 34:     printk(KERN_INFO "ml_probe");
 35: 
 36:   return 0;
 37: } 
 38:  
 39: static void ml_disconnect(struct usb_interface *interface) 
 40: { 
 41:     /* called when unplugging a USB device. */  
 42:     printk(KERN_INFO "ml_disconnect"); 
 43: } 
 44:   
 45: static struct usb_driver fx2_table = { 
 46:     .name = "myFx2", 
 47:     .id_table = ml_table, 
 48:     .probe = ml_probe, 
 49:     .disconnect = ml_disconnect, 
 50: }; 
 51:   
 52: /*  
 53:  * Initialize the module - Register the character device  
 54:  */ 
 55: int init_module() 
 56: {
 57:     int retval;
 58:      
 59:   printk(KERN_INFO "Initializing USB FX\n");
 60: 
 61:     retval = usb_register(&fx2_table);
 62:    
 63:     printk(KERN_ALERT"THE usb_register RET::%d\n",retval); 
 64:     
 65:   return 0; 
 66: }     
 67:  
 68: void cleanup_module() 
 69: { 
 70:      printk(KERN_INFO "cleanup_module\n"); 
 71: }
 72: 
 73: MODULE_LICENSE("GPL");

The make file :

  1: obj-m := usbfx.o 
  2: 
  3: KDIR  := /lib/modules/$(shell uname -r)/build
  4: PWD   := $(shell pwd)
  5: 
  6: default:
  7:     $(MAKE) -C $(KDIR) M=$(PWD) modules

After building the project with make I have tried to install it using the insmod command. and I got the following error:
"insmod: error inserting 'usbfx.ko': -1 Unknown symbol in module"
I tried to figure out the problem using dmseg comamnd .

The log entery of the problem:
[ 3569.172900] usbfx: Unknown symbol usb_register_driver (err 0)

The following article explain about the problem of GPL licensee shortcut.
I had line 73 in order to declare the module license as GPL

Trying to install the module now with insmod  then dmseg produce:

[ 4666.501865] Initializing USB FX
[ 4666.501894] usbcore: registered new interface driver myFx2
[ 4666.501897] THE usb_register RET::0
[ 4700.271816] usb 1-1.1: new high-speed USB device number 4 using ehci_hcd

Now connecting my usb fx device  to the usb port
dmseg produce
[ 4701.740804] ml_probe

cypher variable length relationship

Note the demo db DataBase creation code can be found in prev posts.
Mark with an asterisk (*)
The following example gets all connected nodes in all levels:

  1: theCyperCode = "START Brusc=node(1) MATCH Brusc-[*]->(ConnectedNode) RETURN  ConnectedNode"
  2:          
  3: cypher.execute(graph_db, theCyperCode, row_handler=handle_row)

The results :
cypher query result:
(2 {"name":"John McClane"})
cypher query result:
(5 {"name":"Nakatomi Plaza"})
cypher query result:
(4 {"name":"Hans Gruber"})
cypher query result:
(5 {"name":"Nakatomi Plaza"})
Note that node 5 was visited twice and its apear twice in the result set

We can limit the depth by specifing a range in the form:[*MinDepth..MaxDepth]
for en example

  1: theCyperCode = "START Brusc=node(1) MATCH Brusc-[*2..3]->(ConnectedNode) RETURN  ConnectedNode"
  2: cypher.execute(graph_db, theCyperCode, row_handler=handle_row)

The result:
cypher query result:
(5 {"name":"Nakatomi Plaza"})
cypher query result:
(4 {"name":"Hans Gruber"})
cypher query result:
(5 {"name":"Nakatomi Plaza"})
Only the 2nd and the 3rd level were returned

C++ Tuples

The test class

  1: class MYObj
  2: {
  3: 	public:
  4: 		MYObj(std::string pName);
  5: 	public:
  6: 		std::string mName;
  7: };
  8: 
  9: MYObj::MYObj(std::string pName)
 10: {
 11: 	mName = pName;
 12: }

An example for simple C++ tuple:

  1: 	MYObj myObj;
  2: 	//Create a tuple with 3 different variable types 
  3: 	auto myTuple = std::make_tuple ("Zvika" , 42 , myObj );
  4: 	//Get the first member of the tuple 
  5: 	auto myName = std::get<0>(myTuple);
  6: 	std::cout << myName; 

Use of forward_as_tuple in order to insert to the tuple right values

  1: 	MYObj myObj ("Lion");
  2: 	//Create a tuple with 3 different variable types 
  3: 	auto myTuple = std::make_tuple ("Cat" , 42 , myObj );
  4: 	//Create a tuple with 3 different variable types of right values 
  5: 	auto myTupleAsRightValue = std::forward_as_tuple ("cat" , 42 , myObj );
  6: 	//Chaneg the object state 
  7: 	myObj.mName = "weasel";
  8:    //Get the 3 member of the tuple 
  9: 	auto myName = std::get<2>(myTuple);
 10: 	
 11: 	std::cout << myName.mName <<"\r\n";
 12: 	//Get the 3 member of the tuple of right values 	
 13: 	auto myNameAsRightValue = std::get<2>(myTupleAsRightValue);
 14: 	
 15: 	std::cout << myNameAsRightValue.mName  <<"\r\n";

The result:
Lion
weasel

Declare the tuple without the auto keyword

  1: MYObj myObj ("Lion");
  2: 	//Create a tuple with 3 different variable types 
  3: 	std::tuple <std::string , int , MYObj> myTuple = std::make_tuple ("Cat" , 42 , myObj );
  4: 	//Create a tuple with 3 different variable types of right values 
  5: 	int theVal = 42;
  6: 	std::string theStr = "cat";
  7: 
  8: 	std::tuple <std::string &, int &, MYObj &> myTupleAsRightValue = std::forward_as_tuple (theStr , theVal  , myObj );
  9: 

Reference to Tuple:
http://en.cppreference.com/w/cpp/utility/tuple/forward_as_tuple

MongoDB Cont

Deleting documents from collection .
The following code deletes all documents that the property RunningNo value is 9

  1: import pymongo
  2: from datetime import *
  3: client = pymongo.MongoClient("localhost", 27017)
  4: #Create of get the DB  
  5: db = client['test-database']
  6: 
  7: print (db.name)
  8: 
  9: #create user collection
 10: 
 11: userCollection = db['userCollection']
 12: 
 13: for nextNo in range (1,10,2):
 14: 
 15:     new_user = {"Name":"zvika",
 16:                 "Age":42,
 17:                 "Childs":["Lior","Gal","Shahf"],
 18:                 "dateofbirth" : datetime(1970, 10, 25),
 19:                 "email" : "loveme42@hotmail.com",
 20:                 "RunningNo":nextNo ,
 21:                 "Blog address":"http://zvikastechnologiesblog.blogspot.com"}
 22: 
 23:     userCollection.save (new_user)
 24: 
 25: userCollection.remove ({"RunningNo":9} ,safe=True)
 26: 
 27: users = db.userCollection.find({"Name":"zvika"},{"RunningNo":1,"email":2},sort=[("RunningNo", pymongo.DESCENDING)] ,limit=24)
 28: 
 29: for user in users:
 30:     print ( type ( user))
 31:     print (  user)
 

using sub documents

  1: new_user = {"Name":"zvika",
  2:     "Age":42,
  3:     "Childs":["Lior","Gal","Shahf"],
  4:     "dateofbirth" : datetime(1970, 10, 25),
  5:     "email" : "loveme42@hotmail.com",
  6:     "RunningNo":nextNo ,
  7:     "Blog address":"http://zvikastechnologiesblog.blogspot.com",
  8:     "BankAcounts":[{"Name":"Leumi" ,"No":"123"},{"Name":"apoalim" ,"No":"456"}]}
  9: 
 10: userCollection.save (new_user)
 11: 
 12: users = db.userCollection.find({"BankAcounts.Name":"Leumi"},limit=1)

Note: the code above returns that all  documents that match the query

Setting anew Property

  1: db.userCollection.update({"BankAcounts.Name":"Leumi"},
  2: {"$set":{"NewField":"NewFieldValue"}}, safe=True)

The Result:
{'email': 'loveme42@hotmail.com', 'Name': 'zvika', 'BankAcounts': [{'No': '123', 'Name': 'Leumi'}, {'No': '456', 'Name': 'apoalim'}], 'Childs': ['Lior', 'Gal', 'Shahf'], 'dateofbirth': datetime.datetime(1970, 10, 25, 0, 0), 'Blog address': 'http://zvikastechnologiesblog.blogspot.com', 'RunningNo': 175, 'NewField': 'NewFieldValue', '_id': ObjectId('51a4bcb4def1c124d490d385'), 'Age': 42}

Insert new property to the collection

  1: db.userCollection.update({"BankAcounts.Name":"Leumi"},
  2:     {"$set":{"BankAcounts.$.NewField":"NewFieldValue"}}, safe=True)

note the $ sign is used to represent a collection
The Result:
{'RunningNo': 175, 'BankAcounts': [{'NewField': 'NewFieldValue', 'No': '123', 'Name': 'Leumi'}, {'No': '456', 'Name': 'apoalim'}], '_id': ObjectId('51a4bd67def1c1143c53f8a4'), 'email': 'loveme42@hotmail.com', 'Name': 'zvika', 'Blog address': 'http://zvikastechnologiesblog.blogspot.com', 'Childs': ['Lior', 'Gal', 'Shahf'], 'Age': 42, 'dateofbirth': datetime.datetime(1970, 10, 25, 0, 0)}

Service Mix

Apache Service mix is composed from the following products :
Apache ActiveMQ
Apache Camel
Apache CXF
Apache ODE
Apache Karaf

The installation Guide for ServiceMix Here:

Starting the ServiceMix contaioner:
bin\servicemix.sh

Starting stopping and restarting a java service app in bash

I found the following bash script used to allow stopping restarting and starting of an java application service:

  1: #!/bin/sh
  2: case "$1" in
  3: 
  4: 	start)
  5: 		echo "Start myService"
  6: 		cd /home/ubuntu/myService
  7: 		nohup java -cp .:Service-bin:lib/* com.myCompany.Services.myService > OutputData.log &
  8: 		sleep 3
  9: 		tail -n 10 OutputData.log
 10: 	;;
 11: 
 12: 	stop)
 13: 		echo "Stop service"
 14: 		kill $(ps aux | grep myService|awk '{print $2}') > /dev/null
 15: 	;;
 16: 
 17: 	restart)
 18: 		echo "Restart myService"
 19: 		$0 stop
 20: 		sleep 1
 21: 		$0 start
 22: 	;;
 23: esac
 24: 
 25: exit 0

Things I want to emphasize:
1.The restart case is reenter into the script using the $0
2.Running the service using the nohup bash command .The nohup bash command allows to run command./process or shell script that can continue running in the background after you log out from a shell.
3.The java –cp option allows to provide the classpath i.e. path(s) to additional classes or libraries that your program may require when being compiled or run.
4.Tail –n  10:prints only the final 10 lines of the log
5.The killing of the service line (line 14) list all the current process with the following columns:
USER = user owning the process
PID = process ID of the process
%CPU = It is the CPU time used divided by the time the process has been running.
%MEM = ratio of the process’s resident set size to the physical memory on the machine
VSZ = virtual memory usage of entire process
RSS = resident set size, the non-swapped physical memory that a task has used
TTY = controlling tty (terminal)
STAT = multi-character process state
START = starting time or date of the process
TIME = cumulative CPU time
COMMAND = command with all its arguments 
we try to find the lines with the smyervice phrase and then use awk to get the 2nd word (the Pid that is going to be killed ).
We discard any output to directing the output to the null device .

Camel setHeader

The following camel sample code demonstrates reading text from the console writing it to both to a file and to a console .

  1: try {
  2: 	CamelContext context = new DefaultCamelContext();	
  3: 	context.addRoutes(new RouteBuilder() {
  4: 	public void configure() {
  5: 
  6: 		from("stream:in?promptMessage=Enter something please :").
  7: 		setHeader(Exchange.FILE_NAME, constant("myTxt.txt"))
  8: 		.to("file:target/TestmyFiles").
  9: 			to("stream:out");
 10: 	}
 11: 	});
 12: 	context.start();
 13: 	//Let the camel thread route to work in its thread
 14: 	Thread.sleep(334000);
 15: 	 System.out.print("I'm out");
 16: 			 
 17: } catch (Exception e) {
 18: 	e.printStackTrace();
 19: }

The camel framework auto create the file if it is not exist and override the file content every new route execution  .

UPNP

In order to mimic an hardware device sending a UPNP message we use the windows UPNP Api.
The device simulator that send UPNP messages is build from the UPNPSampleDimmerDevice sample.
The sample is build from 2 parts :
 
The RegDevice is used to send the device registration notification, and the UPNPSampleDimmerDevice is used in order to implement a service that the client could query for the device state .

The UPNP device registration request is done using the following code :

  1: #include <upnphost.h>
  2: 
  3:  IUPnPRegistrar *pReg = NULL;
  4: 
  5:  hr = CoCreateInstance(
  6:       CLSID_UPnPRegistrar,
  7:       NULL,
  8:       CLSCTX_LOCAL_SERVER,
  9:       IID_IUPnPRegistrar,
 10:       (LPVOID *) &pReg
 11:       );
 12: 
 13:    hr = pReg->RegisterRunningDevice(desDoc,punk,initBSTR,resourcePathBSTR,lifeTime,&DeviceID);

The request main parameters are :
desDoc :The device description data .

punk – The device control com interface .

resourcePathBSTR – Path of the directory containing the device description file and the icons

lifetime – The announcements sending cycles in seconds.

The method returns the generated DeviceID

The demo description data :

  1: <?xml version="1.0"?>
  2: <root xmlns="urn:schemas-upnp-org:device-1-0">
  3: <specVersion>
  4:     <major>1</major>
  5:     <minor>0</minor>
  6: </specVersion>
  7: 
  8: <device>
  9:     <pnpx:X_deviceCategory xmlns:pnpx="http://schemas.microsoft.com/windows/pnpx/2005/11">HomeAutomation</pnpx:X_deviceCategory>
 10:     <pnpx:X_hardwareId xmlns:pnpx="http://schemas.microsoft.com/windows/pnpx/2005/11">Microsoft/SampleDevice/10000/urn:microsoft-com:device:SampleDimmerDevice:1</pnpx:X_hardwareId> 
 11:     <df:X_containerId xmlns:df="http://schemas.microsoft.com/windows/2008/09/devicefoundation">{8C7B310D-90F8-40D4-B5EA-71D81821DEA6}</df:X_containerId>
 12:     <deviceType>urn:microsoft-com:device:DimmerDevice:1</deviceType>
 13:     <friendlyName>UPNP SDK Dimmer Device Hosted by Windows</friendlyName>
 14:     <manufacturer>Microsoft</manufacturer>
 15:     <manufacturerURL>http://www.microsoft.com/</manufacturerURL>
 16:     <modelDescription>UPnP SDK Device Host Dimmer Device 1.0 </modelDescription>  
 17:     <modelName>Dimmer1.0</modelName>
 18:     <modelNumber>1-00-00</modelNumber>
 19:     <modelURL>http://www.microsoft.com/</modelURL>
 20:     <serialNumber>0000001</serialNumber>
 21:     <UDN>uuid:RootDevice</UDN>
 22:     <UPC>00000-00001</UPC>
 23: 
 24:     <serviceList>
 25:         <service>
 26:             <serviceType>urn:microsoft-com:service:DimmerService:1</serviceType>
 27:             <serviceId>urn:microsoft-com:serviceId:DimmerService1.0</serviceId>
 28:             <controlURL></controlURL>
 29:             <eventSubURL></eventSubURL>
 30:             <SCPDURL>DimmingService_SCPD.xml</SCPDURL>
 31:         </service>
 32:     </serviceList>  
 33: 
 34:     <presentationURL>DimmerPresentation.htm</presentationURL>
 35: </device>
 36: </root>
 37: 

The upnp service send 4  multicast messgases :

 

 

Creating a device description here

wcf calls thread affinity

A week ago we have notice that many WCF service calls are executing in the UI thread context.
We wish to eliminate this behavior in order to void blocking the UI thread by WCF service calls.
Because the WCF service calls execution time is very long we wants that every call will be execute in a new thread and not using a thread pool .
The solution for this problem was :

Create custom SynchronizationContext that overrides the send and post methods

  1: using System;
  2: using System.Collections.Generic;
  3: using System.Linq;
  4: using System.Text;
  5: using System.Threading;
  6: 
  7: namespace WcfInfra
  8: {
  9:     public class NewThreadForEachCallSyncContext: SynchronizationContext
 10:     {
 11:         public NewThreadForEachCallSyncContext ()
 12: 	    {
 13: 
 14: 	    }
 15:         public override void Send(SendOrPostCallback d, object state)
 16:         {
 17:             new Thread(new ThreadStart(() =>
 18:             {
 19:                 d(state);
 20:             })).Start();
 21:         }
 22:         public override void Post(SendOrPostCallback d, object state)
 23:         {
 24:             new Thread (new ThreadStart (()=>
 25:             {
 26:                 d(state);
 27:             })).Start ();
 28:         }
 29: 
 30:     }
 31: }

Create a wcf service attribute that implements the IContractBehavior .In the applydispatchbehavior attach to the dispatchRuntime.SynchronizationContext  an instance of our custom SynchronizationContext .

  1: public class NewThreadForEachCallInterceptorAttribute : Attribute, IContractBehavior
  2: {
  3: 	public NewThreadForEachCallInterceptorAttribute()
  4: 	{
  5: 
  6: 	}
  7: 
  8: 	public void AddBindingParameters(ContractDescription contractDescription, ServiceEndpoint endpoint, System.ServiceModel.Channels.BindingParameterCollection bindingParameters)
  9: 	{
 10: 		
 11: 	}
 12: 
 13: 	public void ApplyClientBehavior(ContractDescription contractDescription, ServiceEndpoint endpoint, System.ServiceModel.Dispatcher.ClientRuntime clientRuntime)
 14: 	{
 15: 		
 16: 	}
 17: 
 18: 	public void ApplyDispatchBehavior(ContractDescription contractDescription, ServiceEndpoint endpoint, System.ServiceModel.Dispatcher.DispatchRuntime dispatchRuntime)
 19: 	{
 20: 		dispatchRuntime.SynchronizationContext = new TrainerSyncContext();
 21: 	}
 22: 
 23: 	public void Validate(ContractDescription contractDescription, ServiceEndpoint endpoint)
 24: 	{
 25: 
 26: 	}
 27: }

Adore the wcf service with the new attribute

  1: [ServiceBehavior(IncludeExceptionDetailInFaults = true ,InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)]
  2:     [NewThreadForEachCallInterceptor()]
  3:     [ErrorBehavior(typeof(ApplicationException))]
  4:     public class DoThingsLongTimeService : IDoThing