Andrew Cooke | Contents | Latest | RSS | Previous | Next

C[omp]ute

Welcome to my blog, which was once a mailing list of the same name and is still generated by mail. Please reply via the "comment" links.

Always interested in offers/projects/new ideas. Eclectic experience in fields like: numerical computing; Python web; Java enterprise; functional languages; GPGPU; SQL databases; etc. Based in Santiago, Chile; telecommute worldwide. CV; email.

Personal Projects

Choochoo Training Diary

Last 100 entries

Surprise Paradox; [Books] Good Author List; [Computing] Efficient queries with grouping in Postgres; [Computing] Automatic Wake (Linux); [Computing] AWS CDK Aspects in Go; [Bike] Adidas Gravel Shoes; [Computing, Horror] Biological Chips; [Books] Weird Lit Recs; [Covid] Extended SIR Models; [Art] York-based Printmaker; [Physics] Quantum Transitions are not Instantaneous; [Computing] AI and Drum Machines; [Computing] Probabilities, Stopping Times, Martingales; bpftrace Intro Article; [Computing] Starlab Systems - Linux Laptops; [Computing] Extended Berkeley Packet Filter; [Green] Mainspring Linear Generator; Better Approach; Rummikub Solver; Chilean Poetry; Felicitations - Empowerment Grant; [Bike] Fixing Spyre Brakes (That Need Constant Adjustment); [Computing, Music] Raspberry Pi Media (Audio) Streamer; [Computing] Amazing Hack To Embed DSL In Python; [Bike] Ruta Del Condor (El Alfalfal); [Bike] Estimating Power On Climbs; [Computing] Applying Azure B2C Authentication To Function Apps; [Bike] Gearing On The Back Of An Envelope; [Computing] Okular and Postscript in OpenSuse; There's a fix!; [Computing] Fail2Ban on OpenSuse Leap 15.3 (NFTables); [Cycling, Computing] Power Calculation and Brakes; [Hardware, Computing] Amazing Pockit Computer; Bullying; How I Am - 3 Years Post Accident, 8+ Years With MS; [USA Politics] In America's Uncivil War Republicans Are The Aggressors; [Programming] Selenium and Python; Better Walking Data; [Bike] How Fast Before Walking More Efficient Than Cycling?; [COVID] Coronavirus And Cycling; [Programming] Docker on OpenSuse; Cadence v Speed; [Bike] Gearing For Real Cyclists; [Programming] React plotting - visx; [Programming] React Leaflet; AliExpress Independent Sellers; Applebaum - Twilight of Democracy; [Politics] Back + US Elections; [Programming,Exercise] Simple Timer Script; [News] 2019: The year revolt went global; [Politics] The world's most-surveilled cities; [Bike] Hope Freehub; [Restaurant] Mama Chau's (Chinese, Providencia); [Politics] Brexit Podcast; [Diary] Pneumonia; [Politics] Britain's Reichstag Fire moment; install cairo; [Programming] GCC Sanitizer Flags; [GPU, Programming] Per-Thread Program Counters; My Bike Accident - Looking Back One Year; [Python] Geographic heights are incredibly easy!; [Cooking] Cookie Recipe; Efficient, Simple, Directed Maximisation of Noisy Function; And for argparse; Bash Completion in Python; [Computing] Configuring Github Jekyll Locally; [Maths, Link] The Napkin Project; You can Masquerade in Firewalld; [Bike] Servicing Budget (Spring) Forks; [Crypto] CIA Internet Comms Failure; [Python] Cute Rate Limiting API; [Causality] Judea Pearl Lecture; [Security, Computing] Chinese Hardware Hack Of Supermicro Boards; SQLAlchemy Joined Table Inheritance and Delete Cascade; [Translation] The Club; [Computing] Super Potato Bruh; [Computing] Extending Jupyter; Further HRM Details; [Computing, Bike] Activities in ch2; [Books, Link] Modern Japanese Lit; What ended up there; [Link, Book] Logic Book; Update - Garmin Express / Connect; Garmin Forerunner 35 v 230; [Link, Politics, Internet] Government Trolls; [Link, Politics] Why identity politics benefits the right more than the left; SSH Forwarding; A Specification For Repeating Events; A Fight for the Soul of Science; [Science, Book, Link] Lost In Math; OpenSuse Leap 15 Network Fixes; Update; [Book] Galileo's Middle Finger; [Bike] Chinese Carbon Rims; [Bike] Servicing Shimano XT Front Hub HB-M8010; [Bike] Aliexpress Cycling Tops; [Computing] Change to ssh handling of multiple identities?; [Bike] Endura Hummvee Lite II; [Computing] Marble Based Logic; [Link, Politics] Sanity Check For Nuclear Launch; [Link, Science] Entropy and Life

© 2006-2017 Andrew Cooke (site) / post authors (content).

Mule + Java Interfaces

From: "andrew cooke" <andrew@...>

Date: Fri, 13 Jan 2006 15:53:20 -0300 (CLST)

These are some notes I wrote at work to clarify how messaging with Mule
could (should?) work in our system.  One of the side issues here is that
it's unclear what J2EE buys us.

[...]

Consider two services implemented as:

class edu.noao.nsa.msg.SomeMessage {
  // serializable data transfer object with getters/setters
}

interface edu.noao.nsa.serviceA.ServiceA {
  public SomeResponse doSomething(SomeMessage message);
}

class edu.noao.nsa.serviceA.impl.ServiceAImpl implements ServiceA {
  // implements the functionality as a simple bean
}

@Local interface edu.noao.nsa.serviceA.ejb.ServiceALocal
extends ServiceA {}

@Remote interface edu.noao.nsa.serviceA.ejb.ServiceARemote
extends ServiceA {}

@Stateless class edu.noao.nsa.serviceA.ejb.ServiceA
extends ServiceAImpl {
  // do injection from @Resource here
}

class edu.noao.nsa.serviceB.impl.ServiceB {
  private ServiceA serviceA; // plus getters/setters
  public Object someProcess(...) {
    ...
    response = serviceA.doSomething(message);
    ...
  }
}

note the package structure:

edu
 +- noao
     +- nsa
         +- msg
         |   +- serializable message beans
         +- serviceA
         |   +- ServiceA as interface
         |   +- impl
         |   |   +- ServiceA implemented as bean
         |   +- ejb
         |   |   +- wrappers for EJB3
         |   +- test etc
         +- serviceB
             +- ServiceB as interface
             +- impl
             |   +- ServiceB implemented as bean
             +- ejb
                 +- wrappers for EJB3

Now for testing we can write Java unit tests on the impl classes.  We can
even connect together services (using Spring) to test more than one
service at a time.

Note - testing service together may seem more like integration testing,
but in the VOI we follow this same pattern down inside the VOI.  So the
VOI itself is composed of sub-services (components) that plug together as
required.  These then test as described above, but can still be deployed
separately.

The EJB wrappers do two things:
- provide Local/Remote interfaces
- do the necessary injection via @Resource


So as monolithic Java code, that just works.  Either in J2EE or in some
Spring based container.  What extra work needs to be done to deploy the
services on separate machines?

First we need to implement a proxy that intercepts the message from
serviceB.  We already have a set of MessageSender implementations which
send a message to Mule.  So the code is:

class edu.noao.nsa.serviceA.mule.ServiceAMsgProxy implements ServiceA {
  private MessageSender messageSender;
  public void setMessageSender...;
  public MessageSender getMessageSender...;
  public SomeResponse doSomething(SomeMessage message) {
    return (SomeResponse)getSender().sendSynchronous(message);
  }
}


That's it.


Seriously.  No more Java code.  That will send the message object to Mule,
which can be configured to send the message anywhere, via any messaging
system (JMS / convert it to XML / send it as an email / SOAP /
in-memory/VM etc).

This proxy is injected into ServiceB.  ServiceB sees the same interface as
before and talks to it as before.

At the ServiceA end, no extra code is needed at all.  Instead you either:
- deploy ServiceA within Mule, in which case routing is trivial
  (practically automatic)
- deploy ServiceA in EJB, in which case you configure Mule to call the EJB
  (we extend Mule to call EJBs - this was a strange ommission in Mule, but
extension - adding a new "provider" - is quite simple).

So the advantages are:
- Write and test pure Java code
- Type checking of inter-service contracts via interfaces
- No messaging code appears in the "business logic"
- All routing decisions are isolated in Mule
- Only a single, simple proxy at the sender is required
- No interface at all is required at the receiver
- The final deployment structure is not fixed in the design
- Progressively more complex messaging technologies can be configured -
Mule provides synchronous transport over asynchronous messaging

One disadvantage is organising the build so that the cross-dependencies
work OK.  This reflects the cross-dependencies in the contracts that each
service relies on - it's not an artifact of bad engineering, but a
reflection that services do depend on each other via their interfaces, and
that this should be verified as early as possible.

I suspect the best build solution is to use two passes - once to generate
the interfaces and a second time to compile code that depends on all the
interfaces available.

Andrew

[Fwd: (Strictly limited) circular dependencies with M2]

From: "andrew cooke" <andrew@...>

Date: Sat, 14 Jan 2006 11:25:32 -0300 (CLST)

---------------------------- Original Message ----------------------------
Subject: (Strictly limited) circular dependencies with M2
From:    "andrew cooke" <andrew@...>
Date:    Sat, January 14, 2006 11:14
To:      users@...
--------------------------------------------------------------------------

Hi,

I have been asked to configure Maven 2 for our new project at work.  The
project architecture is a set of services, each of which will be a
separate sub-project in Maven 2.

However, each service depends on the *external interfaces* of the services
it connects to.

I know that generally circular dependencies are a "bad thing", so I want
to explain carefully why the dependency we have is not:

 Each service has a contract that it implements.  A calling (dependent)
 service depends on this contract.  If the contract is broken, the system
 will fail.

 Since the contract is critical, it makes sense to test it as early as
 possible.  Preferably at compile time.  We can do this (to a limited
 extent) by expressing the contract as an interface.  So any code that
 uses service X compiles againts the interface that X provides.

 We can then insert proxies and messaging so that the services can exist
 on different machines, etc.  More details at
 http://www.acooke.org/cute/MuleJavaIn0.html

Of course, we do not want "deep" dependencies of one service on another,
so the external interfaces are restricted to:

 - Java interfaces that the service implements
 - Data transfer objects (serializable, simple beans)

How should this be structured?  Since the data transfer objects are shared
between services and have some additional constraints from the messaging
technology (serializable; it would also be useful to have a carefully
designed inheritance hierarchy for simplifying routing decisions based on
class) I think they should go into a separate "messaging" project.  The
interfaces that each service provides, however, belong in the appropriate
service.

So we have the package structure:

 project
  +- messaging
  |   +- data transfer objects
  +- service 1
  |   +- external interface
  |   +- implementation
  +- service 2
  .   +- external interface
  .   +- implementation
  +...

To build this I need to:

 - compile messaging (data transfer objects)
 - compile external interfaces for each service
 - package external interfaces and message objects
   (necessary for the messaging system)
 - compile implementations for each service
 - package each service (both interface and implementation)

(I may also need to package services as ejb3 beans)

If I do this, all dependencies are resolved correctly.

My question, then, is how do I do this with Maven 2?  Sorry I had to write
so much to ask it, and further apopolgies if this is really obvious.  I've
looked around the documentation, but can't see how this level of control
is provided.

Thanks,
Andrew

[Fwd: RE: (Strictly limited) circular dependencies with M2]

From: "andrew cooke" <andrew@...>

Date: Sat, 14 Jan 2006 13:40:37 -0300 (CLST)

---------------------------- Original Message ----------------------------
Subject: RE: (Strictly limited) circular dependencies with M2
From:    "Allison, Bob" <robert.allison@...>
Date:    Sat, January 14, 2006 13:25
--------------------------------------------------------------------------

The normal way of handling this kind of dependency tree is to put the
common stuff (each service's interface and DTOs) is a separate project.
Then each service's project depends on this common stuff project.

Make sense?

[Fwd: RE: (Strictly limited) circular dependencies with M2]

From: "andrew cooke" <andrew@...>

Date: Sat, 14 Jan 2006 13:42:02 -0300 (CLST)

---------------------------- Original Message ----------------------------
Subject: RE: (Strictly limited) circular dependencies with M2
From:    "andrew cooke" <andrew@...>
Date:    Sat, January 14, 2006 13:32
--------------------------------------------------------------------------


It makes sense, but it seems a bit ugly.  I guess there's nothing to stop
the two projects sharing the same Java package namespace.  Hmmm.

OK, I guess I'll go with that.  It seems like Maven is being a bit
intrusive, though.  On the other hand, I guess it will drive home to
everyone the distinction between the two.

Thanks,
Andrew

[Fwd: New, clean, elegeant, succint, correct, consistent, tidy, compact, ordered, directed, cycle-free and generally wonderful maven structure]

From: "andrew cooke (noao)" <acooke@...>

Date: Thu, 23 Feb 2006 19:54:43 -0300 (CLST)

I don't think I'm breaking any rules by forwarding this from work...

---------------------------- Original Message ----------------------------
Subject: New, clean, elegeant, succint, correct, consistent, tidy,
compact,      ordered, directed, cycle-free and generally wonderful maven
structure
From:    "andrew cooke (noao)" <acooke@...>
--------------------------------------------------------------------------

This is how I've left CVS.  PLease shout if it doesn't make sense.

You can get the new tree by doing "cvs checkout nsa-maven".  DON'T try
doing an update into your old tree ;o)


A note on package contents
--------------------------

messaging-iface contains (should contain?  Evan?) message beans (which are
built from simple serializable objects) and interfaces to the services
that messaging implements (messaging itself may be empty at the moment,
but remains as a placeholder in case it is needed in the future).

Other service iface classes define the interfaces that the service
provides, in terms of the message objects.

So a very simple interface looks like:

package edu.noao.nsa.<name>
import edu.noao.nsa.messaging.MyBean

public Interface MyService {
  // argument is a message bean from messaging-iface
  public void doSomething(MyBean msg);
}


Dependencies
------------

This is the dependency tree - code can only refer to code for which there
is a path going *up*.

So, for example, nothing can refer to code in the messaging package.
However, everything except toolkit and mulex can refer to the
messaging-iface package, which contains the message POJOs and the
interfaces that the messaging infrastructure provides.

Another example - the generic <name> package (ie any typical future
package) only has access to <name>-iface (which it implements) and
messaging-iface.  So it can do its own work, create message objects and
send messages (but nothing else).

There is one complication.  The old messaging code was not written with
this structure in mind.  For now, every service depends on messaging-old.
However, this will be phased out once we have scavenged that code for
anything that might be useful and people have eliminated dependencies.

Oh, and "toolkit" is a placeholder for common (service-independent)
utility code, currently empty.

toolkit
|
 \- messaging-iface------\    messaging-old
    |                     |   :
    |\-security-iface----\|   :
    |  |                  |   :
    |   \-security........|...:
    |                     |   :
    |\-dataservice-iface-\|   :
    |  |                  |   :
    |   \-dataservice.....|...:
    |                     |   :
    |\-logging-iface-----\|   :
    |  |                  |   :
    |   \-logging.........|...:
    |                     |   :
    |\-voi-iface---------\|   :
    |  |                  |   :
    |   \-voi.............|...:
    |                     |   :
     \-<name>-iface------\|   :
       |                  |   :
        \-<name>..........|...:   mulex
                          |   :   |
                          \-messaging

[Fwd: Suggested naming standard/structure for a service]

From: "andrew cooke" <andrew@...>

Date: Thu, 23 Feb 2006 19:57:17 -0300 (CLST)

In fact, I think the J2EE remote interface should also be in -iface.

---------------------------- Original Message ----------------------------
Subject: Suggested naming standard/structure for a service
From:    "andrew cooke (noao)" <acooke@...>
-------------------------------------------------------------------------

[not sure anyone but alvaro cares about this, but fyi]

trying to get the security/npa code into a format consistent with
everything we've discussed so far, this is what i think the final "new
proposal alert" code will look like:

in nsa-maven/maven/newproposalalert-iface:
edu
+-noao
  +-nsa
    +-sec
      +-npa
        +-iface
          +-NewProposalAlertIface (interface provided to messaging,
              taking and returning message beans as arguments)

in nsa-maven/maven/newproposalalert:
edu
+-noao
  +-nsa
    +-sec
      +-npa
        +-NewProposalAlert (local interface, expressed in local classes
        |   relevant to internal logic)
        +-iface
        | +-ejb (optional)
        | | +-NewProposalAlertBean (ejb imp. of NewProposalAlertIface,
        | |     has NewProposalAlert implementation hard-wired, perhaps
        | |     by extending NewProposalAlertImpl)
        | +-spring
        |   +-NewProposalAlertBean (spring imp. of NewProposalAlertIface,
        |       receives a NewProposalAlert implementation - an instance
        |       of NewProposalAlterImpl - via spring injection; unpacks
        |       message and calls NewProposalAlert; packages result)
        +-impl
          +-NewProposalAlterImpl (implementation of NewProposalAlert)

in nsa-maven/maven/messaging-iface:
the message beans that appear in NewProposalAlterIface

Comment on this post