2013/11/15

Devoxx 2013 - Conference day 3: Random stuff

Hadoop data mining swiss army knife

Alexandre Dutra Cançado, Bertrand Dechoux, Pablo Lopez
Map/reduce: Hadoop
  • record (key/value) based
  • map/group-by/reduce
  • distributed batch
Hive
  • load data in table
  • sql-like syntax to split lines / do processing (sql can become complex)
  • metastore / HCatalog
  • external tooling (Tableau)
  • partitionng not trivial
  • new language for map/reduce
  • procedural
Cascading:
  • java api
  • procedural

Enterprise Concurrency - practical solutions with JSR236

Alex Heusingfeld refs

Taming Drones: How Java Controls the Uprising of the Drone Force

Eva Veenstra-Kazakov, Timon Veenstra
"specialists at low cost" e.g.
  • rainfall damage - aerial overview
  • biomass estimation -> AgroSense tool
  • track spraying
selled as toy: quadcopter Parrot AR.Drone 2.0
  • library for lowlevel access
  • javafx app (wifi): instructions / data
  • inside / outside options
  • sensors: cameras, sonar, speed ...
  • API AgroSense builds on top of ARDRoneForP5
autopilot through java: very experimental
  • extra raspberry pi (reuse existing usb connector)
  • gps + big antenna - accurate to 2m -> correct with gsm signal (RTK-GPS)
  • challenges: wind, 30 min fly time, mandatory wifi connection, battery, charging, follow tractor...
refs:

Devoxx 2013 - Batch Applications for the Java Platform (JSR 352)

Scott Kurz
JSR352 spec: Batch Applications for the Java Platform
  • JEE7
    • CDI: not required, but used in the ref. spec
  • java6 pre-req
  • SE-friendly (minus transactional req.); but not part of SE...
    • can use other dependency injection frameworks.
implementations (under development!):
  • ref impl: developed exclusively within IBM: readonly for public -- not for production use
  • Spring Batch
  • RedHat
  • Batch EE project Apache (tomee) ???
sequential batch reasons
  • transactional (ACID)
  • efficiency gains
  • massive parallelization not needed
  • facilitate restarts
    • reasons:
      • invalid data
      • lock contentions
      • batch window closed
    • restart:
      • within job: skip completed steps
      • within step: skip processed data-chunks
key concepts (roles)
  • implementor: programming model
    • batchlet: free form
    • chunk: ETL pattern (Extract Transform Load
  • orchestrator:
    • job specification: batch.xml
      • step
      • batchlet
      • support for EL
    • can change depending on implementations!
    • flow control:
      • define exception handling (e.g. skip)
      • interpret and act on return codes
      • retry config (retry with/without rollback)
      • define parallel jobs or steps (aka. partitioning) + support merging (config + programming model (Partion* classes)
  • executor: runtime environment
    • basic components
      • Job Repository
      • JobOperator
    • implementation-specific (not defined in the JSR):
      • clustering, security, ...
      • performance
execution of Job
  • JobInstance
  • JobExecution
  • StepExecution - chunk loop step: one global transaction:
    • ItemReader
      • data access / deserialization of records
      • checkpoint function (positioning)
    • ItemProcessor:
      • core business logic
      • Context gotchas
        • Jobcontext / StepContext: not reliable; consider partitions as running on separate jvms
        • transient data: careful on restart
        • -> design "context" as part of your batch
    • ItemWriter
      • similar to ItemReader, but for output
      • accepts "chunk" of output objects
      • no restrictions on data access
nice first step, looks a bit like EJB 2.x, but, hey, this is only 1.0...:
  • xml config with vendor-specific constructs
  • no really "open" / "usable" ref implementation
  • no "old" and a lot of vendor-specific aspects.
  • no annotations

2013/11/14

Devoxx 2013 - Conference day 2: Random stuff

Go see all the ships in the world

Francesc Campoy Flores
Check out the original talk from Google I/O:
Nice overview of Google Cloud offering and usage

Thalmic Labs - Myo API from init to flush

Gord Tanner, Scott Greenberg
Myo: Gesture Control Armband
details
  • Bluetooth 4.0 Low Energy
  • 9 axis tracking
  • tactile feedback
  • muscle activity sensor
  • Myo API will target multiple languages
  • demo: node.js

Security test automation in software development using open source tools

David Tillemans
ZAProxy -- web application security scanner developed by OWASP

AsciiDoctor

Andres Almiray

NFC + Chrome = Awesomeness

Francois Beaufort
chrome apps:
  • html5 + css3 + js
  • immersive
  • secure
  • offline by default
  • chrome APIs
NFC enabled through USB Chrome API (chrome.usb)
  • USB NFC devices: ACR122U, SCL3711
  • chrome.nfc api
    • findDevices()
    • write()
  • next:
    • bugfix
    • open-source the lib
    • improve performance
Unfortunately not a mainstream browser option for now.

Browser-side security: Mitigate the risk of XSS

Mike West
"lock down applications and securely communicate crossorigin"

Devoxx 2013 - Nuts and Bolts of Java EE 7 Interceptors

Emmanuel Bernard
Interceptor 1.2
  • update of 1.1, part of JEE7
  • common rules
  • used by developers and container providers (e.g. bean validation)
  • AroundConstruct lifecycle
  • priority ranges
Bean Validation and Interceptors
  • define / declare constraints on objects
  • annotation based
  • extensible
  • (new) validation of parameters and return values of methods
  • no proprietary hook
  • ordering of validators -> new @Priority
details
  • @AroundConstruct -> dedicated interceptor class
  • ordering of interceptors:
    • @Priority: global ordering (smaller number = first - cfr. Interceptor.Priority...)
    • or: ordering with @Interceptors list
    • @ExcludeClassInterceptors: remove interceptors defined on classlevel
  • Lyfecycle: simplified method signatures for checked exceptions

Devoxx 2013 - Java Microbenchmark Harness: The Lesser of Two Evils

Aleksey Shipilev
micro-benchmarks:
  • depends on usage
  • aim: explore performance models
java Microbenchmark Harness
issues when running micro-benchmarks
  • warm-up all benchmarks
    • JIT
    • GC config
  • hyperthreading
  • power vs performance logic from cpu -- can change between cores (cpufreq, speedstep,...)
  • OS Scheduler logic
  • time sharing of OS
    • e.g. System.nanoTime() calcs -> de-schedule by OS influences measuring
    • do not overload system
  • avoid dead code (eliminated by the compiler)
    • JMH "BlackHoles"
  • loop unrolling
  • replay / do multiple JVM runs
  • check inlining: -XX:+PrintInlining
  • check memory footprint (SRAM cache vs slow DRAM mem)
  • cpu branch prediction (out-of-order engines) -> realistic data
conclusion: micro-benchmarks are not simple :-)
  • check everything
  • retest multiple times
  • use JMH...

Devoxx 2013 - How to make your Android apps suck. A rant.

Litrik De Roy
Android problems to avoid
  • crazy permissions. E.g.
    • directly call phone numbers
    • phone status & identity: tracking
    • retrieve other apps running
    • mock location for testing -> test-only settings...
  • splash screens :-(
  • "menu button of shame" lower right corner of app
  • Skeumorphic design
    • try to use iOS design on Android
    • tabs at the bottom of the screen
  • spammy notifications: games
  • exit confirmation "are you sure you want to exit?" -> doesn't exist..
  • portrait only mode
  • tabs without swiping
  • Jerky scrolling
  • themed buttons without selectors -> pressed, selected, active , disabled
  • app size (=feature creep)
  • flooding logcat (too many debug messages)

Devoxx 2013 keynote: Shaping the future of web development

Lars Bak, Kasper Lund
Dart is out of beta: 1.0
  • open sourced: www.dartlang.org (BSD License)
  • structured programming language
  • reliable libs
  • IDE (based on eclipse)
  • dart2js translator
  • ongoing development (speed, more compact, standardized etc.)
Dart
IDE -- Dart SDK1.0
  • special Chromium with DartVM integrated
  • integrated debugger
  • automatic update on code changes
  • code completion
Real Dart Projects:
Performance of Dart:
  • best on native Dart VM
  • converted to JS: comparable to JS
  • startup performance imporved because of snapshot mechanism (Dart VM only)
Dart outside the browser:
  • standalone
  • IO libs (networking, file, ...)
  • cpus: ARM, MIPS, x64

2013/11/13

Devoxx 2013 - EJB 3.2 and Beyond

David Blevins
deprecated stuff: (aka. 'optional')
  • Entity Beans (CMP & BMP)
  • JAX-RPC -> replaced by JAX WS and JAX-RS
    • XML-RPC
    • Soap Encoding
API Groups: EJB spec sliced in groups -- check if your JEE server supports your requirements:
  • EJB lite
  • MDB
  • EJB remote
  • persistent Timers
  • JAX-WS
  • Embeddable EJB Container
  • EJB 2.x
  • Entity beans
small fixes:
  • lifted restriction from EJB: "do not touch the file system" (wasn't enforced)
  • business interfaces: allow multiple interface implementation
  • interceptors: difficult method signatures -> 1 method can intercept all events
  • transactional callbacks: now allowed for @Stateful
  • passivation capable (EJB with CDI scopes)
  • timer service: more global, usable outside bean code (javax.ejb.Timer & TimerHandle)
  • security: implicit '**' role (= allow any authenticated user) and no more xml needed
Bigger fix: Message-Driven Beans (MDB)
  • basics: (ongoing!)
    • not specific to JMS! (= Connector-Driven Beans)
    • configuration outdated
      • loosely typed
      • bad documentation
    • static interface (text)
  • learning from JAX-RS
    • no interfaces
    • fluid method signatures
    • annotation-based config
      • strongly typed
      • self-documenting
      • targeted (class, method or param)
  • api for creating your own connector:
    • implement ResourceAdapter: bean invocation
    • MessageEndPoint
    • annotation in MessageDriven bean: @MessageDriven
    • demo: implementation accessible with telnet: custom command prompt / commands.
    • -> MDB is basis for accessing any legacy system
    • -> can replace JMX; portable / standard
  • ref: https://github.com/dblevins/mdb-improvements
future JEE8 (under development):
  • CDI-EJB Alignment
  • scopes are similar
  • fundamentally the same: both return a "fake" Proxy object as interface
  • difference is services: EJB has extra security and transaction -> allow EJB services on CDI beans
  • life-cycle lineup: @Singleton, @Stateful
  • deprecate EJB (will take years...)

Devoxx 2013 - Cryptographic operations in the browser

Nick Van den Bleeken
why
  • easy and secure authentication
  • sign/verify messages & docs
  • encrypt docs
existing crypto frameworks (Dojo, CryptoJS, etc.)
  • completely implemented in JavaScript
  • hard to protect against timing attacks (optimizations in JS engine change)
  • Math.random() is not secure
  • hard to get correct
  • performance issues (e.g. number is 64bit float)
alternatives:
  • java plugins
    • disabled in many browsers (recommendation US DHS)
    • not on mobile
  • native extensions
    • separate install
    • browser specific (some browsers drop support for extension!)
    • not on mobile
solution: Web Cryptography API
  • W3C standard: http://www.w3.org/TR/WebCryptoAPI/ (under construction...)
  • JS API for web developers
  • still requires TLS to prevent sniffing communication
  • still relatively complex (requires correct usage of encryption algorithms)
low level API
  • Basic building block: window.crypto.subtle
    • .generateKey(algorithmKeyGen ...)
      • callback when key is generated
    • importKey: add existing keys (e.g. public key of bank)
    • verify() signature
    • decrypt() message
    • sign() message
    • ...
  • use and combine it for more complex operation
high level API:
  • what framework to choose NaCl, KeyCzar, SJCL or ...
  • hard to define
  • existing APIs should address this: JQuery, Dojo, Prototype etc.
  • -> Not for now...
implemented by
  • MS IE11 (old version of spec)
  • chromium (flags)
  • mozilla (under development)
  • missing: Safari...
  • polycrypt (old version of spec)
  • Netflix: webcrypto key discovery
 future need: smartcard support (but: not a focus for US-based companies...)

Devoxx 2013 - Conference day 1: Random stuff

keynote: Java, Chess, and the Internet of Things

Stephen Chin, Richard Bair
Dukepad
  • http://j.mp/DukePad
  • raspberry pi --> very slow cpu; very good GPU
  • java se embedded (ARM)
  • javafx
demo Chessrobot

Fluentlenium

Mathilde Lemee

WebJars - The Web in JARs


The Crazyflie Nano Quadcopter development platform

Tobias Antonsson, Arnaud Taffanel, Marcus Eliasson
http://www.bitcraze.se/


Taming Functional Web Testing (with Spock & Geb)

Luke Daley, Peter Niederwieser
to check...

Arquillian testing framework

Devoxx 2013: The Modular Java Platform and Project Jigsaw

Mark Reinhold
Java platform layers; total of 54 mb:
  • compact 1: 11mb: core
  • compact 2: 16mb (Jdbc, rmi, jaxp,...)
  • compact 3: 30mb (rowset, jdbc, prefs, xmldsig)
  • 54mb: javafx, crypto, httpservers, corba...
  • + extra tools (jaxws, devtools,...)
Not a perfect solution -> better to make a dependency model of each component (directed graph).
Performance solution needed for:
  • fast class lookup
  • aggressive inlining
  • ahead of time compilation (startup speed)
  • JVM-specific memory images
  • removal of unused fields/methods/classes
Solution:Jlink (optional tool)
  • input: *.jar, *.class, *.jmod
  • output: jmod, jar, JVM image
  • e.g.:
    • jlink -mods javafx,rmi,nashorn -d $JRE
    • javac -mods javafx,rmi,nashorn -d classes *.java
    • java -mods javafx,rmi,nashorn -jar ...
  • requires changes to the Java Runtime: no rt.jar & tools.jar anymore (bad representation for JVM)
Security & modularity
  • current setup can't hide some parts. E.g.: sun.misc.Unsafe
  • historic checks: java.lang.SecurityManager::checkPackageAccess: no static checks possible
  • modules & dependency graph can help validate access to sensitive packages
  • some internal packages will be removed -> must be fixed!:
    • sun.*
    • *.internal.*

Devoxx 2013: HTTP Caching in Practice

Xavier Coulon
types of cache:
  • local cache
  • proxy cache
  • gateway cache
  • (app cache -- custom built in the app)
http1.1 caching GET requests
  • expiration Last-Modified + Expires headers
  • If-Modified-Since header -> 304 not modified
  • ETag header
  • If-Match & 412 Precondition failed.
prevent inconsistencies: use explicit directives
  • Cache-Control
    • request header: no-cache, max-age, no-store, max-stale, min-fresh, no-transform & only ..
    • response: public, private, no-cache, proxy-revalidate, max-age,...
  • pragma header, request & response: no-cache: only for http 1.0 backwards compatiblity.
  • Refresh button in brower adds Cache-Control and Pragma headers to request. (not in Ajax requests!
java helpers:
  • Servlet Filter
  • JAX-RS 2.0: interceptors: Ajax-specific (e.g. add ETag + send Status.NOT_MODIFIED)
  • testing:
    • Arquillian (+ REST extension)
    • REST-assured
Https:
  • proxy caching doesn't work
  • decide on using http for static resources...
caching strategy:
  • static content:
    • max cache usage
    • enable gateway caching: Cache-Control: public, max-age
  • dynamic content:
    • timeout or ETag
RFC 2616 spec (http 1.1): http://www.ietf.org/rfc/rfc2616.txt

Devoxx 2013 - Lambda: A Peek Under the Hood

Brian Goetz
lambda expression
  • "anonymous method"
  • type inference compiler
  • capture values from context
why lambdas for java
  • path to multicore support in libraries
  • library developers
  • competition other languages
  • inner classes: clunky
  • @runtime support
typing of lambda expressions
  • function types? -> VM / bytecode issues -> too complex
  • functional interfaces:
    • single method interfaces (cfr Runnable, Comparator)
    • compiler identifies functional interface
    • compiler infers lambda
      representation
  • inner classes? -> performance issues, error-prone, -> "obvious but wrong"
  • MethodHandle in Java7 (tool for compiler writers) -> performance issues, conflates binary representation with impl. -> wrong
  • invokedynamic
    • lets some language logic determine call logic
    • performant impl. in jvm
lambda & invokedynamic
  • code generation strategy (metafactory)
    • dynamic inner classes (=current Java 8 startegy)
    • wrapper class per interface
    • dynamic proxies (or MethodHandleProxy
  • vm-private APIs
    • build object from scratch
  • lazy evaluation
    • non-capturing lambda's: create single instance
    • defer initialization cost to first use
    • VM optimizes call
  • faster than inner classes
  • usable by other languages
  • serialization support: SerializeLambda
  • future VM optimizations
    • escape analysis

Devoxx 2013 Keynote: Java 8 and beyond

Mark Reinhold, Brian Goetz
Java: 18 years old
  • static language, but many dynamic features
  • many great features: gc, JIT ..
  • principles Java Language
    • reading code more important than writing
    • simplicity
    • one language, same everywhere
Java 8 features:
  • lambda expressions (+ type inference)
  • default methods in interfaces
  • java.util.stream: aggregate / data-parallel operations
lambda expressions:
  • inspiration: How to think about Parallel Programming -- Not (Guy Steele) (http://www.infoq.com/presentations/Thinking-Parallel-Programming)
  • more expressive code
    • instead of boilerplate of anonymous inner classes
    • syntax inspired from C#
    • single-method interfaces
    • ´´default´´ methods
      • default implementation
      • fully virtual
      • how stream() method was added
  • more powerful:
    • treat behavior as data
    • "gentle turn for the functional"
    • push away from mutative / imperative
    • huge impact on API design
  • more parallel
    • aggregate operations (filter, conversions, max, etc.)
    • stream() -> parallelStream() (single parallel pass on datasets)
    • retrofitted Collections
next:
  • value types: key enabler for
    • tuples,
    • user-defined primitive types